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(57) Abstract 

A method and apparatus for providing force feedback to a user 
ooerating a human/computer interface device in conjunction with a 
OTphical user interface (GUI) displayed by a host computer system. 
A Dhvsical object, such as a joystick or a mouse, controls a graphical 
obSrsuch afa cursor, within the GUI. Tlie GUI allows the user to 
intcrfic with operating system functions implemented by the computer 
system A signal is output from the host computer to the interface device 
to apply a force sensation to the physical object using one or more 
actuatore. This desired force sensation is associated with at least one 
of the graphical objects and operating system functions of the grapn»«;a 
user interface and is determined by a location of the cursor in the GUI 
with respect to targets that arc associated with the graphical objects. The 
graphical objects include icons, windows, pull-down menus and menu 
items scroll bars ("sliders"), and bunons. TTic force sensation assists the 
user to select a desired operating system function or physically infonns 
the user of the graphical objects encountered by the cursor within the 
GUI. A microprocessor local to the interface apparatus and separate from 
the host computer can be used to control forces on the physical object. 
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Pg THnnANin apparatus for PEO >^TnTNr, Foprp ff,f.dback 

FOR A GRAPHICAL IL S FR INTERFACE 

Tprhnical Field 

The present invention relates generally to interface devices for allowing humans to interface 
with computer systems, and more panicularly to computer systems and computer interface devices 
that provide force feedback to the user. 

po/;}fgr9""d An 

Computer systems are used extensively in many different industries to implement many 
applications, such as word processing, data management, simulations, games, and other tasks. 
These types of applications are very popular with the mass market of home consumers. A 
computer system typically displays a visual enviromnent to a user on a display screen or other 
visual output device. Users can interact with the displayed environment to perform functions on 
the computer, play a game, experience a simulation or "virtual reality" environment, use a 
computer aided design (CAD) system, or otherwise influence events or images depicted on the 
screen. Such user interaction can be implemented through the use of a human-computer mterface 
device, such as a joystick, mouse, trackball, stylus and tablet, "joypad" button controller, foot 
pedal, yoke hand grip, or the like, that is connected to the computer system controlhng the 
displaced environment. The computer updates the environmem in response to the user's 
manipulation of an object such as a joystick handle or mouse, and provides feedback to the user 
utilizing the display screen and, typically, audio speakers. 

One visual environment that is particularly common is a graphical user interface (GUI). 
Information within GUI's are presented to users through purely visual and auditory means such as 
a video monitor and sound card to present images and sound effects which describe various 
graphical metaphors of the operating system. Common GUI's include the Windows® operating 
system from Microsoft Corporation and the System 7 operating system from Apple Computer. Inc. 
These interfaces allows a user to graphically select and manipulate functions of the operating 
system of the computer by using a mouse, trackball, joystick, or other input device. Other 
graphical computer environments are simUar to GUI's. For example, graphical "pages" on the 
World Wide Web of the Internet communication network utilize features similar to that of GUI's to 
select and operate particular functions. Some CAD systems similarly provide graphical 
presentations. In addition, there has been some contemplauon of three dimensional (3-D) GUI's 
that present simulated 3-D environments on a 2-D screen. 
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GUI's typically require users to carefully move and position a user-controlled graphical 
object, such as a cursor or pointer, across the screen and onto other displayed graphical objects or 
predefined regions on a computer screen. Such manual tasks can be described as "targeting" 
activities where a user physically manipulates a mouse, joystick, or other interface device in order 
to command the cursor to a desired location or displayed object, known as a "target" herein. Such 
targets can include, for example, icons for executing application programs and manipulaung files: 
windows for displaying icons and other information; pull-down menus for selecting particular 
functions of the operating system or an application program; buttons for selecting presented 
options; and scroll bars or "sliders" for scrolling information in windows. 

Upon moving the cursor to the desired target, the user must maintain the cursor at the acquired 
target while pressing a button, squeezing a trigger, depressing a pedal, or making some other gesture 
,0 command the execution of the given selection or operation. Examples of targeting tasks include 
positioning a cursor on a graphical icon, selecting and pressing a graphical representation of a button, 
choosing among numerous items within a graphical representation of a pull-down menu, settmg a 
15 continuous analog value from a provided range of values by positioning an indicator within a 
graphical representation of a scroll bar, selecting a region of text by highlighting a region using the 
cursor, as well as a number of other common windows-based and text-based metaphors. 

The movement of a cursor onto various displayed graphical objects of a GUI may require 
significant dexteritv. Users may move the cursor too far over an object and have to backtrack their 
20 cursor. Or. particular graphical objects might be mistakenly selected when the user does not wish to 
select the object due to pressing a button or moving the cursor by accident. In addition, a user may 
become confused as to which window a cursor is positioned in if the user is viewing other data on the 
screen at the same time as moving the cursor. 

In particular, persons with neuromotor disabiliUes who suffer from spastic manual conuo! 
25 have much greater difficulty interacting with GUIs because they lack the fine motor coordination 
required to manually position the computer cursor accurately and efficiently. While manual targeung 
activities are adequately executed by persons with normal neuromotor functionality, persons with 
spastic hand motions find such tasks to be physically challenging if not impossible. 

What is needed is a computer system and interface device that will allow all users to more 
30 accurately and efficiently perform cursor movement activities and manipulate operating system and 
other functions within a GUI. 
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Fri'Tln'^"'^^ Invention 

The present invention is directed to controlling and providing force feedback to a user 
operating a human/computer interface device in conjunction with a graphical user interface (GUI 
displayed by a host computer system. Force sensations are provided to the interface d-cc to ass.st 
5 anL infom. the user of graphical objects encountered by a user.ontroUed cursor m the GUI. 

More specifically, a method of the present invention for providing force feedback within a 
graphical user interface (GUI) environment of a computer system includes a step of recemng an 
indLtion of movement of a physical object that is manipulated by a user. This physical object, 
such as a joystick handle or a mouse, is included in an interface device that outputs the md.cauon 
10 of movement to the computer .system". A user-controUed graphical object, such as a cur.sor. .s 
„,cved within a graphical user interface (GUI) based on the indication of the movement of the 
phys:cal object. Preferably, a position control paradigm is implemented such that the locauon of 
the cursor in the GUI approximately corresponds to a location of the physical object wuh refe^nce 
to an origin; alternatively, a rate control paradigm may be used. The cursor and the GUI are 
1 5 displayed on a display screen connected to the computer system, and the GUI allows O^e user .0 
interface w.th operating system functions implemented by the computer system through grap^c^ 
objects displayed on the screen. A signal is output from the computer system to the mrerfac 
device to command the interface device to apply a des.red force sensation to the phys.cal object 
using one or more electrically controlled actuators. This desired force sensation >s assoc.ated w.th 
20 at least one of the graphical objects and operating system functions of the graphical user mterface. 

Preferably, the force sensation applied to the physical object is at least partially detennined 
bv a location of the cursor in the GUI with respect to targets associated with the graphical objects 
in the GUI These targets may include or be associated with such graphical objects as .cons, 
windows, puU-down menus and menu items, scroll bars ("sliders"), and buttons. The force 

.5 sensation output to the physical object is associated with targets that affect the cursor. Thas force 
preferably assists the user to select the desired operating system function that is associated w.th the 
force For example, a target can provide an attractive force on the physical object and cursor so 
that the cursor is more easily moved onto the target. In addition, the force on the physical object 
may inform the user of the graphical object that the cursor has moved into or near. An operaung 

30 svstem function may be performed as indicated by the location of .he cur.sor and as md.catcd by a 
command from the user, such as a physical (or simulated) button press. Velocity or acceleration of 
the cursor may also affect the applied force. 

Each of the targets is preferably as.sociated with at least two different target force sensations 
that may affect the physical object and the cursor depending on the location of the cursor with 
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respect to each target. The two different target force sensations include an internal target force 
sensation and an external target force sensation. The internal target force is applied to the physical 
object when the cursor is located within or moving in or out of the target. The external target force 
is applied to the physical object when the cursor is located outside the target. Tbe targets are also 
preferably ordered in a hierarchy, and a target's level in the hierarchy dctemunes if the target will 
provide forces on the physical object. 

The magnitude, direction, duration, and other parameters of the internal and external forces 
of a target can depend on the type of the target. For example, the external force sensation of icons 
is an attractive force between the icon and the cursor, which is applied to the physical object when 
the cursor is within a predetermined distance of the icon. An internal capture force of an icon is 
preferably an attractive force when the cursor is moved into the icon, and a barrier force when the 
cursor is moved out of the icon. An internal dead region force is preferably zero near the center 
area of the icon so the cursor can be moved freely when inside the icon. OUier graphical objects 
can be assigned forces in desired ranges within and external to the graphical objects. A damping 
15 force can be used as a dead region force for other graphical objects to provide resistance to the 
motion of the physical object. In addition, an inenia force can be applied to the physical object 
when a target is moved by the cursor in the GUI. The target can have a simulated mass that allows 
a resistive force to be applied to the physical object based on the mass, velocity, or other factors. 

A system of the present invention for providing force feedback to a user manipulating an 
20 interface apparatus includes a host computer system. The host receives an input signal from the 
interface apparatus describing the location, velocity and/or acceleration of the physical object in a 
degree of freedom. The host provides a host output signal and updates the location of the cursor 
within the GUI displayed on the display screen based on die input signal. A microprocessor local 
to the interface apparatus and separate from the host receives the host output signal and provides a 
25 processor output signal. An actuator receives the processor output signal and provides a force 
along a degree of freedom to the physical object in accordance with the processor signal. A sensor 
detects motion of the physical object along the degree of freedom and outputs the input signal 
including information representative of Jie motion of the physical object. Preferably, the sensor 
outputs the input signal to the local microprocessor, which outputs the input signal to the host. 
30 The physical object can preferably be moved in one or more degrees of freedom using, for 
example, a gimbal or slotted yoke mechanism, where an actuator and sensor can be provided for 
each degree of freedom. A standard serial interface included on many computers, such as the 
Universal Serial Bus. can be used to interface the host computer system with the local 
microprocessor. A clock is preferably coupled lo the host computer system and/or the local 
35 processor which can be accessed for timing data to help determine the force output by the actuator. 
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Th= ho.« computer can n^ccivc sensor infom>:«ion in a supervisory mode and ou.po, a 
Mgh ieve, hos. command ,„ ^ microprocessor w^never a force sensation feU by " ^ 

^^ated or changed, .naccordanc. wirK .be hos. con,»nd. .he ""-'"^^ 
Ing dau and o..puu force values «, U» ac«.a,or accordtng ,o a force sensauo P-«y^ 
selecL The force sensation process can include using force =,ua.,nns, readmg force profUcs o 
ItHed force values from a storage device, or other steps, and may be dependent on se sor 
Clgl'^ost command data, or other data. Memanveiy. the hos, can din=ct,y contro, the 

actuators of the interface device. 

,n another method of the present invenUon for providing force feedback for graphical 
Objects ,n a game implemented on a computer system, a user<ontro,led firs, graphtca, objeu or 
-oaddle" is dtsplaved on a display screen of tf,e compurer syslcm. The paddle moves on 

d^l bvruser, A Lond graphical object or "bal,- is also displayed and -ed on the d.splay 
^n When the paddle collides with ^ ball, a compression of the paddle is d,sp,ayed at the 
Where the ball contacts ^e paddle. The paddle and ball each have a p^detemun^ 
llated mass and/or s,mu,a.ed compliance. A force command is also "^'^'^^ J"^ 
device .0 apply a force to ,he physical objec, in a, leas, one degree of freedo,,,. The fo ce .s app^d 
„ . e direcuon of the compress.on and has a magnttude ,n accordance with the 
compliances, and velocities of ,he graphical objects. In addition, factors such as gravty c^ affec. 
the movement of the graphical objects on the screen and the feces applied to the physical object. 

The method and apparatus of the present invenUon advantageously provtdes force feedback 
,0 a user in conjunction with movement of a cursor in a GUI. Th,s allows the movemen, of a,e 
cursor to be affected by forces output on the physical objcc, manipulated by ,hc user. Thus, me 
forces can assist in manipulating operating sysum funcions of the GUI anCor info^ the u.,cr o^ 
the OU, spati^ "landscape- of graphical objects, providing a more efficient OUL /^so. phys,c.^ 
tandicap^ users have a far easier time moving a cursor ,o various graphic^ ohjecs and r=gK» 
of a GUI when the forces of the present invention are provided In addition, a separate 
microprocessor local to the .nterface device can read and process sensor signals as well as ot-tpu 
force command signals independently of the host computer, thus saving significan, processing ..me 
on ,he hos, compu,er and providing more accura,e force feedback when using a senal or other 
relatively low-bandwidih communica.ion inKrface benveen .he hos. and .he imerface devce. 

These and other advantages of the present invem.on will become apparcn, ,o ,hose skilled 
in , he an upon a reading of the following specir,ea.ion of ,he invemion and a sludy of ,he several 
figures of the drawing. 
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pripf Description nf thf. Drawings 

Figure 1 is a block diagram of a control system in accordance with the present invention for 
controlling a force feedback interface device from a host computer; 

5 Figure 2 is a schematic diagram of an actuator interface for providing control signals to an 

active actuator for the present invention; 

Figure 3 is a schemauc diagram of an actuator interface for providing control signals to a 
passive actuator for the present invention; 

Figure 4 is a now diagram illustrating a first embodiment of a method of the present 
10 invention for controlling a force feedback interface device; 

Figure 5 is a How diagram illustrating a second embodiment of a method of the present 
invention for controlling a force feedback interface device; 

Figure 6 is a schematic diagram of a closed loop five bar Unkage mechanism for providing 
two degrees of freedom to the user object of the interface device; 

1 5 Figure 7 is a perspective view of a preferred embodiment of the linkage mechanism shown 

in Figure 6; 

Figure 8 is a perspective view of a slotted yoke joystick embodiment of a mechanical 
interface for the user object; 

Figure 9 is a table summarizing rate control commands of the present invention; 
20 Figures 1 Oa-c are diagrammatic representations of restoring force profiles; 

Figures 1 la-c are diagrammatic repr.sentations of restoring spring force profiles; 
Figure 12 is a diagrammatic representation of a vector force; 
Figures 13a-b are diagrammatic representations of vibration force profiles; 
Figure 14 is a table summarizing position control commands of the present invention; 
25 Figure 15 is a diagrammatic representation of a groove force profile; 

Figure 16 is a diagrammatic representation of a barrier force profile; 
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Figurcs 1 7a-17i are diagrammatic illustrations of a paddle and ball imeracuon controlled by 
a paddle command of the present invention; 

Figuies 17j-k are diagrammatic illustrations of paddle and ball embodiments displayed on a 
display screen; 

Figure 18 is a diagrammatic illustration of a display screen showmg a graphical user 
interface (GUI) and the interaction of forces of the pre.«.nt invention with a user-controlled cursor; 

Figure 19 is a diagrammatic illustration of a display screen showing the GUI of Figure 18 
having three windows with forces affectmg the user-controlled cursor; 

Figure 20a is a diagrammatic illustration of a display screen showing targets of a GUI and 
external forces provided by those targets; 

Figure20bisadiagrammatic illustration ofatarget and the internal forces provided by that 

target; 

Figure 20c is a diagrammatic illustration of another embodiment of a target and external 
forces for that target; 

Figure 2 1 is a diagrammatic illustration of a display screen showing the GUI of Figure 1 8 
having a pull-down menu and associated forces of the present invention; 

Figure 22 is a diagrammatic illustration of a display screen showing the GUI of Figure 18 
havmg a pop-up window with buttons and associated forces of the present invention; 

Figure 23 is a flow diagram illustrating a method of the present invention for providing 
force feedback within a GUI; 

Figure 24 is a flow diagram illustrating a step of Figure 23 for assigning force ranges and 
magnitudes to graphical objects with a GUI; 

Figure 25 is a flow diagram iUusiratmg a step of Figure 23 for detem^inrng the target of 
lowest hierarchy in which the cursor is positioned; 

Figure 26 is a flow diagram illustrating a step of Figure 23 for applying appropriate forces 
to the user object based on targets in the GUI; and 

Figure 27 is a flow diagram illustrating a method for applying external and internal forces 
for a target based on the position of the cursor. 
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Rp,^t MnHpR for Cany inp out the Invention 

FIGURE 1 is a block diagram illustrating a generic control system 10 of the present 
invention for an interface device controlled by a host computer system. Control system 10 
5 includes a host computer system 12 and an interface device 14. 

Host computer system 12 is preferably a personal computer, such as an IBM -compatible or 
Macintosh personal computer, or a workstation, such as a SUN or Silicon Graphics workstation. 
For example, the host computer system can a personal computer which operates under the MS- 
DOS or Windows operating systems in conformance with an IBM PC AT standard. Alternatively. 
10 host computer system 12 can be one of a variety of home video game systems commonly 
connected to a television set, such as systems available from Nintendo. Sega, or Sony. In other 
embodiments, home computer system 12 can be a "set top box" which can be used, for example, 
to provide interactive television functions to users. 

In the described embodiment, host computer system 12 implements a host application 
15 program with which a user 22 is interacting via peripherals and interface device 14. For example, 
the host application program can be an operating system, graphical user interface (GUI), video 
game, medical simulation, scienufic analysis program, or even an operating system or other 
application program that utilizes force feedback. Typically, the host application provides images to 
be displayed on a display output device, as described below, and/or other feedback, such as 
20 auditor)' signals. 

Host computer system 12 preferably includes a host microprocessor 16, random access 
memory (RAM) 17, read-only memory (ROM) 19, inputyoutput (I/O) electronics 21, a clock 18, a 
display screen 20, and an audio output device 21. Host microprocessor 16 can include a variety of 
available microprocessors from Intel, Motorola, or other manufacturers. Microprocessor 16 can be 
25 single microprocessor chip, or can include multiple primary and/or co-processors. Microprocessor 
preferably reuievesand stores instructions and other necessary data from RAM 17 and ROM 19, 
as is well known to those skilled in the art. In the described embodiment, host computer system 
12 can receive sensor data or a sensor signal via a bus 24 from sensors of interface device 14 and 
other information. Microprocessor 16 can receive data from bus 24 usmg I/O electronics 21, and 
30 can use I/O electronics to control other peripheral devices. Host computer system 12 can also 
output a "force command" to interface device 14 via bus 24 to cau.se force feedback for the 
interface device. Clock 18 is a standard clock crystal or equivalent component used by host 
computer system 12 to provide liming to elecuical signals used by microprocessor 16 and other 
components of the computer system. Clock 18 is accessed by host computer system 12 in the 
3,*5 control process, as described subsequently. 
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Display screen 20 is coupled .o host microprocessor 16 by suuable display drivers and can 
be used to display images generated by host computer system 12 or other --P^-J-^ 
Display screen 20 can be a standard display screen or CRT. 3-D goggles or any o her v^a 
interfi In a described embodiment, d.splay screen 20 displays ,mages of a GUU apph auon 
si.ulat.on or game environment. In other embod.ments. other -^2^^:^ 
A user 22 of the host computer 12 and mterface device 14 can rece.ve v.sual feedback by v.ev^mg 

display screen 20. 

Herein compulcv 12 may be refcned as displaying compmcr or graphical "objects- or 
-.ntife" These compurer objects are not physical objects, bu, is a logical software umt 
<:::L,.ofdataana/or procedures that .ay be displayed as intakes by — ^ ^ 
screen 20. as is well known to those skilled ,n the an. Fo, example, a cursor or a thtrd-person 
;ie„„fac.nygh.beco«side.dpl.yer.controlledco„.pu,er objects that can be moved acr^^^^ 

screen. A displayed, sitnulated cockpit of an ^cratt r^ght also be constdered an object . or the 
simulated aircraft can be considered a computer controlled "entity' . 

Audio output device 2 1 . such as speakers, is preferably coupled to host mic«.processor 16 
v,a ampbners, filters, and other ctrcuiuy well known to those skilled in the an Host processor .6 
outputs signals to speakers 21 to ptovide sound output to user 22 when an atri.o event t^cur^ 
duing the ,mplem«.tation of the host application program. Other types of pcnpherals can ^dso be 
IplL to ht^st processor 16. such as storage devices (hard di.sk drive. CD ROM dnve. floppy 
20 disk drive, etc.). printers, and other input and output devices. 

An interface device 14 is coupled to host computer system 12 by a bi-directional bus 24. 
The bi-directional bus sends signals in either direction "oetween ho.« computer system 12 and the 
interface device. Herein, the term "bus" is intended to genetically refer to an interface such as 
between host computer 12 and microprocessor 26 which typically includes one or more connecung 
wire, or other connections and that can be implemented in a variety of ways, as descnbed below 
In the preferred embodiment, bus 24 is a serial interface bus providing data -cording to a senal 
communication protocol. An interface port of host computer sys.^-. 12, such as an RS23. sertal 
interface port, connects bus 24 to host computer system 12. Other standard senal communicauon 
protocols can also be used in the serial interface and bus 24. such as RS-422. Universal Serial Bus 
(USB) MIDI, or other protocols well known to those skilled in the art. For example, the USB 
standard provides a relatively high speed serial interface that can prov.de force feedback signals in 
the present invention with a high degree of realism. USB can also source more power to dnve 
peripheral devices. Since each device that accesses the USB is assigned a unique USB address by 
the host computer, this allows multiple devices to share the same bus. In addition, the USD 
35 standard includes liming data that is encoded along with differential data. 
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An advantage of the present invention is that low-bandwidth serial communication signals 
can be used to interface with interface device 14. thus allowing a standard built-in serial interface of 
many computers to be used directly. Alternatively, a parallel port of host computer system 12 car 
be coupled to a parallel bus 24 and communicate with interface device using a paraUcl protocol, 
such as SCSI or PC Parallel Printer Bus. In a different embodimeni, bu.s 24 can be connected 
directly to a data bus of host computer system 12 using, for example, a plug-in card and slot or 
other access of computer .system 12. For example, on an IBM AT compatible computer, the 
interface card can be implemented as an ISA. EISA. VESA local bu.s. PCI. or other well-known 
.standard interface card which plugs into the motherboard of the computer and provides input and 
output ports connected to the main data bus of the computer. 

In another embodiment, an additional bus 25 can be included to communicate between host 
computer system 12 and interface device 14. Bus 24 can be coupled to the standard serial port of 
host computer 12. while additional bus 25 can be coupled to a second port of the host computer 
system. For example, many computer systems include a "game port" in addiuon to a serial RS- 
232 port to connect a joystick or similar game controller to the computer. The two buses 24 and 25 
can be used simultaneously to provide a increased data bandwidth. For example, microprocessor 
26 can send sensor signals to host computer 12 via a uni-dircctional bus 25 and a game port, while 
host computer 12 can output force feedback signals from a serial port to microprocessor 26 via a 
uni-directional bus 24. Other combinations of data flow configurations can be implemented in 
20 other embodiments. 

Interface device 14 includes a local microprocessor 26. sensors 28. actuators 30. a user 
object 34. optional sensor interface 36. an optional actuator interface 38. and other optional input 
devices 39. Interface device 14 may also include additional clccuonic components for 
communicating via standard protocols on bus 24. In the preferred embodiment, multiple interface 
25 . devices 14 can be coupled to a .single host computer system 12 through bus 24 (or multiple buses 
24) so that multiple users can simultaneously interface with the host application program (in a 
multi-player game or simulation, for example). In addition, multiple players can interact in the host 
application program with multiple interface devices 14 using networked host computers 12. as is 
well known to those skilled in the art. 

30 Local microprocessor 26 is coupled to bus 24 and is preferably included wii'nin the housing 

of interface device 14 to allow quick communication with other components of the interface device. 
Processor 26 is considered "local" to interface device 14, where "local" herein refers to processor 
26 being a separate microprocessor from any processors in host computer system 12. "Local" also 
preferably refers to processor 26 being dedicated to force feedback and sensor I/O of interface 

35 device 14. and being closely coupled to sensors 28 and actuators 30. such as within the housing 
for interface device or in a housing coupled closely to interface device 14. Microprocessor 26 can 
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be provided with software mstructions to wa.t for commands or requests from computer host 16. 
decode the command or request, and handle/control ,nput and output signals accordmg to the 
command or request. In addition, processor 26 preferably operates '"^^^^ 
computer 16 by reading sensor signals and calculating appropnate for.es from those senso 
5 signals, tirr. signals, and a force sensation proces.s (also referred to as a •subrouune herem) 
selected in accordance with a host command. Su.table microproce.ssors for use as lo^al 
microprocessor 26 include the MC68HC71 1E9 by Motorola and the PIC16C74 by M.croch.p. for 
example Microprocessor 26 can include one microprocessor chip, or multiple processors and/or 
co-processor chips. In other cmbod:mcnts. microprocessor 26 can includes a dig>ul stgnal 
,0 processor (DSP) ch.p. Microprocessor 26 can receive signals from sensors 28 and P-'de -gna^. 
to actuators 30 of the interface dev.ce 14 in accordance with in.structions provided by host 
computer 12 over bus 24. Microprocessor 26 can also receive commands from any other mput 
devices included on interface apparatus 14 and provides appropriate signals to ho.st computer 12 to 
.ndicate that the mput tnfomiation has been received and any infomtation included .n the mpu. 
15 information. For example, buttons, switches, dials, or other input controls on interface dev.ce 
or user object 34 can provide signals to microprocessor 26. 

Local memory 27. such as RAM and/or ROM, is preferably coupled to microprocessor 26 
in interface device 14 to store instnictions for microprocessor 26 and store temporary and oAer 
data, such as calibraUon parameters. A local clock 29 can be coupled to the microprocessor 26 to 
.0 provide ummg data, similar to system clock 18 of host computer 12; the timing data might be 
' Lquired. for example, to compute forces output by actuators 30. In alternate embodiments using 
the USB communication interface, timing data for microprocessor 26 can be retrieved from a USB 
signal. 

In one embodiment, host computer 12 can provide low-level force commands over bus 24, 
25 which microprocessor 26 directly provides to actuators 30. This embodiment is descnbed m 
greater detail with respect to Figure 4. In a different embodiment, host computer system 12 can 
provide high level supervisory commands to microprocessor 26 over bus 24. and microprocessor 
26 manages low level local force control loops to sensors 28 and actuators 30 m accordance u.th 
the high level commands. This embodiment is described in greater detail with respect to Figure 5. 

30 In the preferred embodiment, sensors 28, actuators 30, and microprocessor 26, and other 

related electronic components are included in a housing for interface device 14. to which user 
object 34 is directly or indirectly coupled. Alternatively, microprocessor 26 and/or other elcctronu: 
components of interface device 14 can be provided in a separate housing from user object 34, 
sensors 28. and actuators 30. Also, additional mechanical stmctures may he included m mterface 

35 device 14 to provide object 34 with desired degrees of freedom. Some embodiments of such 
mechanisms are described with reference to Figures 6-8. 
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Sensors 28 sense the position, motion, and/or other characteristics of a user object 34 of 
the interface device 14 along one or more degrees of freedom and provide signals to 
microprocessor 26 including information representative of those characteristics. Examples of 
embodiments of user objects and movement within provided degrees of freedom are described 

5 subsequently with respect to Figures 6-8. Typically, a sensor 28 is provided for each degree of 
freedom along which object 34 can be moved, or a single compound sensor can be used to sense 
movement in multiple degrees of freedom. An example of sensors suitable for several 
embodiments described herein are digital optical encoder.s. which sense the change in position of 
an object about a rotational axis and provide digital signals indicative of the change in position. 

1 0 Linear optical encoders similarly .sense the change in position of object 34 along a linear degree of 
freedom. Either relative or ab.solute sensors can be u.sed. A suitable optical encoder is the 
"Softpot" from U.S. Digital of Vancouver. Washington. 

Sensors 28 provide an electrical signal to an optional sensor interface 36, which can be 
used to convert sensor signals to signals that can be interpreted by the microprocessor 26 and/or 
15 host computer system 12. For example, sensor interface 36 receives the two phase-related signals 
from a sensor 28 and converts the two signals into another pair of clock signals, which drive a bi- 
directional binary counter. The output of the binarj' counter is received by microprocessor 26 as a 
binary number representing the angular position of the encoded shaft. Such circuits, or equivalent 
circuits, are well known to those skilled in the art; for example, the Quadrature Chip LS7I66 from 
20 Hewlett Packard. California performs the functions described above. Each sensor 28 can be 
provided with its own sensor interface, or one sensor interface may handle data from mulUplc 
sensors. For example, a sensor interface can include a separate processing chip dedicated to each 
sensor thai provides input data. Altcmatcly, microprocessor 26 can perform these interface 
functions without the need for a separate sensor interface 36. The position value signals can be 
25 used by microprocessor 26 and are also sent to host computer system 12 which updates the hast 
application program and outputs force control signals. In alternate embodiments, sensor signals 
from sensors 28 can be provided directly to host computer system 12. bypassing microproces.sor 
26. Also, sensor interface 36 can be included within host computer system 12. such as o . an 
interface board or card. 



30 



Alternatively, an analog sensor can be used instead of digital .sensor for all or .some of the 
sensors 28. For example, a su-ain gauge can be connected to measure forces on object 34 rather 
than positions of the object. Also, velocity sensors andyor accelerometers can be used to directly 
measure velocities and accelerations on object 34. Analog sensors can provide an analog signal 
representative of the position/velocity/acceleralion of the user object in a particular degree of 
35 freedom. An analog to digital converter (ADC) can convert the analog signal to a digital signal that 
is received and interpreted by microprocessor 26 and/or host computer system 12. as ts well 
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can include one or more buttons provided, for example, on the joystick handle or base and used to 
supplement the input from the user to a game or simulation. The operation of such input devices is 
well known to those skilled in the art. 

Power supply 40 can optionally be coupled to actuator interface 38 and/or actuators 30 to 
provide electrical power. Acuve actuators typically require a separate power source to be driven. 
Power supply 40 can be included within the housing of interface device 14, or can be provided as a 
separate component, for example, connected by an electrical power cord. Alternatively, if the USB 
or a similar communication protocol is used, interface device 14 can draw power from the USB 
and thus have no need for power supply 40. This embodiment is most applicable to a device 14 
having passive actuators 30. since passive actuators require litUe power to operate. Acuve 
actuators tend to require more power than can be drawn from USB. but this restriction can be 
overcome in a number of ways. One way is to configure interface 14 to appear as more than one 
peripheral to host computer 12; for example, each provided degree of freedom of user object 34 
can be configured as a different peripheral and receive its own allocation of power. This would 
allow host 12 to aUocate more power to interface device 14. Alternatively, power from the USB 
can be stored and regulated by interface device 14 and thus used when needed to drive actuators 
30. For example, power can be stored over time and then immediately dissipated to provide a jolt 
force to the user object 34. A capacitor circuit, for example, can store the energy and dissipate the 
energy when enough power has been stored. Microprocessor may have to regulate the output of 
forces to assure that time is allowed for power to be stored. This power storage embodiment can 
also be used m non-USB embodiments of interface device 14 to allow a smaller power supply 40 
10 be u.sed. 

Safety switch 41 is preferably included in interface device to provide a mechanism to allow 
a user to override and deactivate actuators 30, or require a user to activate actuators 30. for safety 
reasons. Certain types of actuators, especially active actuators such as motors, can pose a safety 
issue fortheuser if the actuators unexpectedly move user object 34 against the user with a strong 
force. In addition, if a failure in the control sy.stem 10 occurs, the user may desire to quickly 
deactivate the actuators to avoid any injury. To provide this option, .safety switch 41 is coupled to 
actuators 30. In the preferred embodiment, the user must contmually activate or close safety 
.switch 41 during operation of interface device 14 to activate the actuators 30. If. at any time, the 
safety switch is deactivated (opened), power from power supply 40 is cut to actuators 30 (or the 
actuators arc otherwise deactivated) as long as the safety switch is deactivated. For example, a 
preferred embodiment of safety switch is an optical switch located on user object 34 (such as a 
joystick) or on a convenient surface of a housing enclosing interface device 14. When the user 
covers the opucal switch with a hand or finger, the sensor of the switch is blocked from sensing 
ambient light, and the switch is closed. The actuators 30 thus will function as long as the user 
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coverstheswitch. Other types of safety switches 41 can be provided m other embodiments. For 
example, an electrostatic contact swUch can be used to sense contact, a button or tngger can be 
pressed, or a different type of sensor switch can be used. 

User object 34 .s preferably a device or article that may be grasped or otherw.se contacted 
or controlled by a user and which . coupled to interface device 14. By -grasp". i^J^ -ant ^a 
users may releasably engage a grip portion of the object in some fash.on. such as by ha d. wU 
the.r nngertips. or even orally m the case of handicapped persons. The user 22 can mantpulate and 
le th! ob^ct along provided degrees of freedom to interface with the hcst ^^^^^^^ 
.he user is v.ewmg on display screen 20. Object 34 can be a joy.stic. mouse track s ylu . 
steering wheel, medical instilment (laparoscope, catheter, etc.). pool cue. hand grtp, knob, 
button, or other article. 

FIGURE 2 is a schematic diagram illustrating an example of an actuator interface 38 for an 
active actuator 30 of interface dev.ce 14. In ^is example, actuator 30 is a linear --t c— 
servo motor. Actuator interface 38 includes a DAC citcu.t 44 and a power amphfi r c r u 46^ 
DAC crcuit 44 is coupled to microprocessor 26 and preferably receives a digital signal 
representing a force value from the nucroprocessor 26. DAC 48 is suitable for converung ^ mpu 
digital signal to an analog voltage that is output to power amplifier circmt 46. Op amp 50, fo^ 
example, outputs a signal from zero to -5 volts proportional to the binary number at its input. Op 
amp 52 is an inverting summing amplifter that converts the output voltage to a symntetr^cd bipo ^ 
range; this output signal is suitable for power amplification in amplificauon circuit 46. Of course, 
DAC circuit 44 is intended as one example of many po.ssible circuits .hat can be used to convert a 
digital signal to a desired analog signal. 

Power amplifier circuit 46 receives an analog low-power control voltage from DAC circuit 
44 and amplifies the voltage to control actuators 30. Actuator 30 can be a high-power, current- 
controlled servo motor 30. The input voltage controls a transconductance stage composed 
amplifier 54 and several resistors. The transconductance stage produces an output current 
proportional to the input voltage to drive rhotor 30 while drawing very little current from the mput 
voltage source. The second amplifier stage, including amplifier 56. resistors, and a capacitor C 
provides additional current capacity by enhancing the voltage swing of the second lermmal 57 o^ 
motor 30. Of course, circuit 46 is intended as one example of many possible circuits that can be 
used to amplify voltages to drive active actuators 30. 

FIGURE 3 IS a schematic diagram illustrating an example of an actuator interface 38' that 
can be used in conjuncaon with passive actuators. Interface 38' is suitable for use with passive 
actuators (dampers) that are comrolled with an analog voltage. Interface 38' includes a DAC c.n:u.t 
5 44 amphfter 60. transistor 62, and voltage protector 64. DAC circuit 44 is coupled to 
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microprocessor 26 and receives a digital signal from the computer system representing a resistive 
force value to be applied to user object 34. DAC circuit 44 converts the digital signal voltages to 
analog voltages which are then output to amplifier 60. Amplifier 60 receives the analog voltage 
from DAC 44 on a positive terminal and scales the voltage signal to a range usable by actuator 30. 
Amplifier 60 can be implemented as an operational amplifier or the like. Transistor 62 is coupled 
10 the output of amplifier 60 and preferably operates as an amplifier to provide increased output 
current to actuator 30. Resistor Rl is coupled between amplifier 60 and the emitter of transistor 
62, and resistor R2 is coupled between amplifier 60 and ground. Voltage protector 64 is coupied 
to the emitter of transistor 62 and provides protection from voltage spikes when usmg inductive 
loads. Suitable passive actuators 30 for use with this circuitry includes variable solenoids or 
magnetic particle brakes. A separate. DAC and amplifier can be used for each actuator 30 
implemented in the interface apparatus. Interface 38' is intended as one example of many possible 
circuit-s that can be u.sed to interface a computer system to actuators. In an alternate embodiment, 
an on/off signal might only be needed, for example, for a solenoid driving an on/off valve of a 
5 fluid-controlled actuator. 

nOURE 4 is a flow diagram illusu-aling a first embodiment of a method 70 for controlling 
a force feedback interface device of the present invention. Method 70 is directed to a "host- 
controlled" embodiment, in which host computer system 12 provides direct, low-level force 
conuuands to microprocessor 26, and the microprocessor directly provides these force commands 
0 to actuators 30 to control forces output by the actuators. For example, the host controlled mode is 
suitable for embodiments using a USB communication interface. Data rates arc sufficiently high to 
allow the host to communicate at 500 Hz or greater and provide realistic force feedback to the user 
object 34. 

The process begins at 72. In step 74, host computer system 12 and interface device 14 arc 
>5 powered up. for example, by a user activating power switches. After step 74, the process 70 
branches into two parallel (simultaneous) processes. One process is implemented on host 
computer system 12, and the other process is implemented on local microprocessor 26 These two 
processes branch out of step 74 in different directions to indicate this simultaneity. 

In the host computer .system process, .step 76 is first implemented, in which an appUcation 
30 program is processed or updated. This appUcation can be a simulation, video game, scientific 
program, or other program. Images can be displayed for a user on output display screen 20 and 
other feedback can be presented, .such as audio feedback. 

Two branches exit step 76 to indicate that there arc two processes running simultaneously 
(multitasking) on host computer system 12. In one process, step 78 is implemented, where sensor 
35 data is received by the host computer from local microprocessor 26. As detailed below in the 
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oucroprocessor process, the local processor 26 contmually receives signals from ser^sors 28 
proceLs the raw data, and sends processed sensor data to host computer 12 Alter^anveiy, loc^ 
processor 26 sends raw data directly to host computer system 12. "Sensor data , as re erred o 
herein can include position values, velocity values, and/or acceleration values derived from the 
sensor's 28 which detect motion of object 34 in one or more degrees of freedom. In add.t.on any 
other data received from other input devices 39 can also be received by host computer system 12 as 
sensor data in step 78, such as signals indicaUng a button on interface device 14 has been ncuvated 
by the user. Fmally, the term "sensor data" also can include a history of values, such as pos.t.on 
values recorded previously and stored in order to calculate a velocity. 

After sen.sor data is read in step 78, the process returns to step 76. where the host computer 
system 12 can update the application program in response to the user's manipulations of object 34 
L any other u.ser input received in step 78 as well as determine if forces need to be applied to 
object 34 in the parallel process. Step 78 is implemented in a continual loop of reading data from 
local processor 26. 

i -me second branch from step 76 is concerned with the process of the ho.st computer 

determining force commands to provide force feedback to the user manipulating object 34. T^ese 
commands ar^ described herein as "low-level" force commands, as distinguished from the high- 
level" or supervisory force commands described in the embodiment of Figure 5. A low level force 
command instructs an actuator to output a force of a particular magnitude. For example, the low 
3 level command typically includes a magnitude force value, e.g., equivalent signal(s) to mstnict ^e 
actuator to apply a force of a desired magnitude value. Low level force commands may also 
designate a direction of force if an actuator can apply force m a selected direction, and/or other low- 
level information as required by an actuator. 

The second branch starts with step 80, in which the host computer system checks if a 
5 change in the force applied to user object 34 is required. This can be determined by several types 
of criteria, the most important of which ^ the sensor data ..ad by the host computer in step 78. 
limine data, and the implementation or "events" of 'He application program updated in step 76. V^c 
sensor data read in step 78 informs the host computer 12 how the user is interacting with the 
application program. From the position of object 34 sensed over time, the host computer system 
30 P can determine when forces should be applied to the object. For example, the position of a 
computer generated object within a GUI may detemiine if a change in force feedback ,s called for. 
In addition, the velocity and/or acceleration of the user object can inHuence whether a change in 
force on the object is required. Also, other input, such as a user activating buttons or other 
controls on interface device 14. can change the forces required on object 34 depending on how 
35 those controls have been programmed to affect the application program. 
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Other criteria for determining if a change in force is required includes events in the 
application program, such as collision events between graphical objecu. Forces should thus be 
applied to the user object dependent on this collision event to simulate an impact. Forces can be 
required on the user object depending on a combination of such an event and the sensor data read 
in step 78. Other parameters in the application program can determine if a change in force to the 
user object is necessary, such as other input devices or user interface devices connected to host 
computer system 12 and inputting data to the application program (other interface devices can be 
directly connected, connected remotely through a network, etc.). 

If no change in force is currently required in step 80, then the process returns to step 76 to 
update the host application and return to step 80 to again check until such a change in force is 
required. When such a change is required, step 82 is implemented, in which host computer 12 
determines appropriate low-level force commands to be sent to the actuators 30 of interface device 
14. these force commands being dependent on a selected force sensation process, .sensor data, the 
host application, and the clock 18. 

The low-level force commands can be determined, in part, from a selected force sensation 
process. A "force sensation process", as referred to herein, is a set of instructions for providing 
force commands dependent on other parameters or conditions, such as sensor data read in step 78 
and liming data from clock 18. In the described embodiment, force sensation processes can 
include several different types of steps and/or instructions. One type of instrucuon is a force 
algorithm, which includes an equation that host computer 12 can use to calculate or model a force 
value based on sensor and timing data. Several types of algorithms can be used. For example, 
algorithms in which force varies linearly (or nonlineariy) with the position of object 34 can be used 
to provide a simulated force like a spring. Algorithms in which force varies linearly (or 
nonlineariy) with the velocity of object 34 can be also used to provide a simulated damping force or 
other forces. Algorithms in which force varies linearly (or nonUnearly) with the acceleration of 
object 34 can also be used to provide, for example, a simulated inertial force on a mass (for linear 
variation) or a simulated gravitational pull (for nonlinear variation). 

For force values depending on the velocity and acceleration of user object 34, tlie velocity 
and acceleration can be provided in a number of different ways. The sensor data read by host 
computer 12 in step 78 can include position data, velocity data, and acceleration data. In a 
preferred embodiment, the velocity and acceleration data was calculated previously by 
microprocessor 26 and then provided to the host computer 12. The host computer can thus use the 
velocity and acceleration data directly in an algorithm to calculate a force value. In an alternate 
embodiment, the sensor data read in step 78 includes position data and no velocity or acceleration 
data, so that host computer 12 is required to calculate the velocity and acceleration from the 
position data. This can be accomplished by recording a number of past position values, recording 
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the time when each such position value was received using the system clock 18, and calculaung a 
velocity and/or acceleration from such data. For example, a kinemauc equation which calculates a 
force based on the velocity of the user object multiplied by a damping constant can be used to 
determine a dampmg force on the user object. This type of equation can simulate motion of object 
34 along one degree of freedom through a fluid or similar material. A damping constant can 
indicate the degree of resistance that object 34 experiences when moving through a simulated 
material, such as a liquid, where a greater number indicates greater resistance. The difference m 
position and direction of movement of the user object is calculated, and the force is set equal to the 
damping constant multiplied by the change in position. Movement in other mediums, such as on a 
0 bumpy surface, on an inclined plane, etc., can be simulated in a similar fashion using different 
methods of calculating the force. 

The determination of force commands is preferably influenced by timing data accessed 
from system clock 18. For example, in the damping force example described above, the velocity 
of the user object 34 is determined by calculating the different of positions of the user object and 
5 multiplying by the damping constant. The host computer can access clock 12 to determine how 
much time has actually elapsed since the last position data was received and thus use the clock's 
timing data in the modulation of forces and force sensations to the user. Timing data can be used 
in other algorithms and force sensation processes of the present invention. 

Other instructions can also be included in a force sensation process. For example, 
>0 condiuons can be included to provide forces only in desired directions or under other particular 
circumstances. For example, to simulate a virtual obstruction such as a wall, forces should be 
applied in only one direction (uni-dircctional). To simulate uni-directional resistance, conditions 
can be included in the obstruction force sen.sation process. Also, a "null" force sensation process 
can be available that instnicts host computer 12 (or microprocessor 26 in Figure 5) to issue a low 
25 level command or force values to provide zero forces (i.e. . remove all forces) on object 34. 

Another type of force sensation process does not use algorithms to model a force, but 
instead uses force values that have been previously calculated or sampled and stored as a digitized 
"force profile" in memory or other storage device. These force values may have been previously 
generated using an equation or algorithm as described above, or provided by sampling and 

30 digitizing forces. For example, to provide a particular force sensation to the user, host computer 
12 can be instructed by a force sensation process to retrieve successive force values from. a certain 
storage device, such as RAM, ROM, hard disk. etc. These force values can be sent directly to an 
actuator to provide particular forces without requiring host computer 12 lo calculate the force 
values. In addition, previously-stored force values can be output with respect lo niher parameters 

35 to provide different types of forces and force sensations from one set of stored force values. For 
example, using system clock 18. the stored force values can be output in sequence according to a 
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particular lime interval that can vary depending on the desired force. Or, different retrieved force 
values can be output depending on the current position of user object 34. 

Host computer 12 can determine a force command in step 82 according to a newly-selected 
force sensation process, or to a previously selected force sensation process. For example, if this is 
5 a second or later iteration of step 82, the same force sensation process as in the previous iteration 
can be again implemented if parameters (such as the position of object 34) allow it. as determined 
by the host application program. 

The force command determined in step 82 can also depend on instructions that check for 
other parameters. These instructions can be included within or external to the above-described 

10 force sensation processes. One such parameter are values provided by the implemented host 
application program (if any). The application program may detemfune that a particular force 
command should be output or force sensation process implemented based on events occurring 
within the application program or other instructions. Force commands or values can be provided 
by the host application program independently of sensor data. Also, the host application program 

15 can provide its own particular position, velocity, and/or acceleration data to a selected force 
sensation process to calculate or provide a force that is not based on the manipulation of user object 
34, but is provided to simulate an event in the applicauon program. Such events may. include 
collision events, such as occur when a user-controlled computer image impacts a virtual surface or 
structure. Also, other input devices connected to host computer 12 can influence events and, 

20 therefore, the forces applied to user object 34. For example, the sensor data from multiple 
interface devices 14 connected to a single host computer can influence the forces felt on other 
connected interface devices by influencing events and computer-controlled images/objects of the 
host application program. Also, the force commands determined in step 82 can he based on other 
inputs to host computer 12, such as activations of buttons or other input devices in (or external to) 

25 interface device 14. For example, a particular application program might require that a force be 
applied to a joystick whenever a user presses a fire button on the joystick. 

The above-described force sensation processes and other parameters can be used to provide 
a variety of haptic sensations to the user through the user object 34 to simulate many different types 
of tactile events. For example, typical haptic sensations may include a virtual damping (described 

30 above), a virtual obstruction, and a virtual texture. Virtual obstructions arc provided to simulate 
walls, obstructions, and other uni-directional forces in a GUI, simulation, game, etc. and are used 
to provide a physical resistance to movment of the joystick in a direction. Vinual textures can be 
used to simulate a surface condition or similar texture. For example, as the user moves a joystick 
along an axis, the host computer sends a rapid sequence of conmiands to repetitively 1 ) apply 

35 resistance along that axis, and 2) to then immediately apply no resistance along that axis, as 
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according .0 a fo.e «n.a,,on process and coTcla^d wi* sparial posUion. Thus, *= u.r fcc,s . 
scnsarion of .exru^, sWia, .o f=e>i.g of dragg-n, a sUc. over a graung^ 

,„ ,„x. ,.ep 81 3 lowleve, force command d«cnmned in s,ep 82 is oulpa. » 
Tover bos 24 Th,s force conm>and W^V "udes a force value *a, was 
^croprocesso ov„ bo parameters descHbed above. The force con™,and c^ i« ou.p« 
determined ,n ^ 30 by microp,oces.scr 26; or, .he for« 

'''''':""Zl T^ rom, by nucroprocessor 26 before being sen, .0 

"r^o 30 in dd nte iowJ. force co^nand preferably includes infom,a.ion ind,ca„ng 
ac,na.or30 In a d mon. ^^^^ ^.^ 

.0 microprocessor ^6 »""ch ac uat _^ p„.css/upda,e >h= host 

i„c,aded on inrerft^ ^ ^^^^^^^ ,„ gO, where .he hos. compo.=r checks „ a 

app,ica.,on P-^'^a nl^ouX outpu. as dcermined by .he paramcers described above. If so. 
different force command should t>e ouipu required, host 

:::ter::e::rr::.h .h^ same force scnsa.ion pr^ess. or a diffe„n, for. 
sensation process, depending on Uie parameters of s.ep SI. 

,„ addition, the host computer ,2 preferably synchronizes any appropnate 
and,to.yf=edbac..orotherfeedb..re,atedtothehostapplica,ionw,thteapplK^^^^^^^^^^^^^^ 

u „ la examolc in a video game application, the onset or start of visual events, s 
r:X— w:htuserondfsp,ayscree„20.s.uldbe.^^^^^^^ 
Stan of orces felt by the user which corrcs^nd to or comp,e,r.nt those visua, ~ 
vi^al events and force events are preferably occur wi.hin abou. 30 m,il,.seconds (ms) of «ch 
. 0:^^ 4 span of tin. is the typtcal ,u»t of human ...eptua, abtli. u, P--^--; 
simultaneous. If the visual and force events occur ouuide this range the a ume lag t^me^n 
vents can usually be perceived. Sinularly. the ^tput of auditoty stgnals, "-^»»^' ^ 'o ^ 
orsetofaudttory events in the hos, appiicaaon. are preferably ou.pu. synchrom^ed wtth ,^ n.^ 
Ipu, forced, co.espond ,o,comp,eme„, ,hosc audi,o^ even,s 
,0 evems occur preferably wi,h.n abou, 30 ms of each o,her. Fo, example, hos, compuu U 
can output sounds of an explosion from speakers 21 as close in ^ ^ 

by the user from that explosion m a simulation. Preferably, the magnitude o. the sound « 
(as opposed ,0 inverse, proporrion ,0 ,he magni,ude of ,he forces applied ,0 user object 34. 

The local microprocessor 26 implements die process branching from s,ep 74 and s,ar,mg 
35 wi,h s,ep 86 in parallel with ,he hos, computer process desctibcd above. In step 86. die mtcrface 
rl 14 s acLted. For example, signals can be sent between host computer .2 ar. interface 
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device 14 to acknowledge that the interface device is now active. From step 86. two processes 
branch to indicate that there are two processes running simultaneously (multi-tasking) on local 
processor 26. In one process, step 88 is implemented, in which the processor 26 reads raw dat;i 
(sensor readings) from sensors 28. Such raw data preferably includes position values describing 
5 the position of the user object along provided degrees of freedom. In alternate embodiments, 
sensors 28 can include velocity sensors and accelerometers for providing raw velocity and 
acceleration values of object 34. The raw data read in step 88 can also include other input, such as 
from an activated button or other control 39 of interface device 14. 

In next step 90. processor 26 processes the received raw data into sensor data, if 
10 applicable. In the preferred embodiment, this processing includes two .steps: computing velocity 
and/or acceleration values from raw position data (if velocity and/or acceleration are needed to 
compute forces), and filtering the computed velocity and acceleration data. The velocity and 
acceleration values are computed from raw position data received in step 88 and stored position and 
time values. Preferably, processor 26 stores a number of position values and time values 
15 corresponding to when the position values were received. Processor 26 can use its own or a local 
system clock (not shown in Fig. 1) to determine the timing data. The velocity and acceleration can 
be computed using the stored position data and timing data and then filtered to remove noise from 
the data, such as large spikes that may result in velocity calculations from quick changes in position 
of object 34. In an alternate embodiment, circuitry that is eleclrically coupled to but separate from 
20 processor 26 can receive the raw data and determine velocity and acceleration. For example, an 
application-specific integrated circuit (ASIC) or discrete logic circuitry can use counters or the like 
to determine velocity and acceleration. 

.^tematively. step 90 can be omitted, and the processor 26 can provide raw position data to 
host computer 12 (and other input data from other input devices 39). This would require host 

25 computer 12 to filter and compute velocity and acceleraUon from the position data. In other 
embodiments, the filtering can be performed on host computer 12 while the velocit>' and 
acceleration calculation can be performed on the processor 26. In embodiments where velocity 
and/or acceleration sensors are used to provide raw velocity and acceleraUon data, the calculation of 
velocity and/or acceleration can be omitted. After .step 90. step 91 is iniplemenied, in which the 

30 processor 26 sends the processed sensor daia to the host computer 12 via bus 24. The process 
then returns to step 88 to read raw data. Steps 88. 90 and 91 are thus continuously implenriented to 
provide current sensor data to host computer system 12. 

The second branch from step 86 is concerned with processor 26 controlling the actuators 
30 to provide forces calculated by host computer 12 to object 34. The second branch starts with 
35 step 92, in which processor 26 checks if a low-level force command has been received from host 
computer 12 over bus 24. If not. the process continually checks for such a force command. When 
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a force co,nma«. has been ^e,ved, s.ep 94 Is imp,em=n,e<i, i„ wh.ch processor 26 cu,pu,s a low- 
. 7™ fo«c command .o .he design... ac,ua.o. ,0 se, .he ctpm force ,0 de 
level P"«««" „ay be equivalen, .0 the reeved lo»-level 

— r:::— r.or..beprc.essor.c.^ 

form usable bv actuator 30 (or actuator interface 38 can penonn 
' IZITZ ;isstr rcL .0 s,ep n ,0 che. ,or ano..r force command fro. ^ 

host computer 12. 

FIGURE 5 is a flow diagram illustrating a second embodiment of a method 100 for 
. r HKnnW interface device 14 of the present invcnuon. Method 100 is directed to a 

,0 ':2t:z:t:zt::c^.^ .si .ovrdes o.. ...... s.^r.... 

10 reflex cmto^n«: . n,icroprocessor 26. while ,he microprocessor 

;::;:erd::ei^san::rdes,Uve,forceco 

an independen. -reflex" to control forces outpm by the actuators. 

The process of Figure 5 is suit^le for lo» speed communicadoo interfaces, such as a 
, , stand jRS.23Ts^..al Interface, Ho„ever, the embcdin„nt of Hgure 5 is also suita«e for h,g 
reTclunicatton interfaces such as USB, since iocal microprocessor reUeve 

sZtforward command protocol, an example of «h,ch is described .,.h respect to F.gur s a^ 
4 L »hich allow software developers to easily provide force feedback in a hos, apphcat.on. In 
20 ,";Tllent, for example, the slower ■■interrupt data transfers- mode of USB can be used. 

The process begins a. 102. In step 104, hos. computer system 12 and interface device 14 
.e powe d up. for example, by a user xnvaung power switches. After step .04. the process 
hrlhes lo two par^le. processes. One process is tmplcmented on host computer system 
12, and the other process is implemented on local microprocessor 26, 
,5 m the host computer system process, s,ep 106 is Ars, implemented, ,n which an applicadon 

' program Is processed. This application can be a simulation, video game, scicntif.c progran. ot 
other progrl. Images can be displayed for a user on output d.splay screen 20 and other feedback 
can be presented, such as audio feedback. 

Two branches exit step 106 to indicate that .he« are two processes runmng simuhane^sly 
30 (multitasking, etc.) on host computer system 12. In one of P'-'""' /''^ J 
implememed. where sensor data from *e user obiec. is received by the hos, computer from local 
microprocessor 26. Similarly to step 78 of the process of Figure 4. hos. computer system 
receives either raw dau (e.g., position data and no velocity or accelenfion data) or proces«d 
sensor data (posiUon, velocity and/or acceleration data) from microprocessor 26. In addtuon, any 
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Other data received from other input devices 39 can also be received by host computer system 12 
from microprocessor 26 in step 108. such as signals indicating a button on interface device 1 4 has 
been pressed by the user. 

Unlike the previous embodiment of Figure 4. the host computer does not calculate force 
5 values from the received sensor data in step 108. Rather, host computer 12 monitors the sensor 
data to detemune when a change in the type of force is required. This is described in greater detail 
below. Of course, host computer 12 also uses the sensor data as input for the host application to 
update the host application accordingly. 

After sensor data is received in step 108, the process returns to step 106. where the host 
10 computer system 12 can update the application program in response to the user's manipulations of 
object 34 and any other user input received in step 108. Step 108 is then implemented again m a 
continual loop of receiving sets of sensor data from local processor 26. Smce the host computer 
docs not need to directly control actuators based on sensor data, the sensor data can be provided at 
a much lower speed. For example, since the host computer updates the host application and 
15 images on display screen 20 in response to sensor data, the sensor data need only be read at 60-70 
Hz (the refresh cvcle of a typical display screen) compared to the much higher rate of about 500- 
1000 Hz (or greater) needed to realisucally provide low-level force feedback signals from sensor 
signals. Host computer 12 also preferably synchronizes visual, audio, and force events similarly 
as described above with reference to Figure 4. 

20 The second branch from step 106 is concerned with the process of the host computer 

determining high-level force commands ("host commands") to provide force feedback to the user 
manipulating object 34. The .second branch starts with step 110, in which the host computer 
system checks if a change in the type of force applied to user object 34 is required. The "type" of 
force is a force sensation or profile produced by a particular force sensation process or force value 
25 which the local microprocessor 26 can implement independently of the host computer. The host 
computer 12 determines whether a change in the type of force is required depending on the sensor 
data read ^-y the host computer in step 108 and depending on the events «)f the application program 
updated in step 106. As explained with reference to Figure 4. the sensor data informs the host 
computer when forces should be applied to the object based on the object's current position. 
30 velocity, and/or acceleration. The user's manipulations of object 34 may have caused a new type 
of force to required. For example, if the user is moving a virtual race car within a virtual pool of 
mud in a video game, a damping type of force should be applied to the object 34 as long as the race 
car moves within the mud. Thus, damping forces need to be continually applied to the object, but 
no change in the type of force is required. When the race car moves out of the pool of mud, a new 
35 type of force (i.e. a removal of damping force in this case) is required. The events of the 
application program may also require a change in the type of force applied. Forces may be 
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^ on .te .ser objcc depeoding on a con*ina.ion of » appLcanon ev=m and ft. «nso, d..a 
I P .OS. A J. o*« inpu.. such a., a u.e, aclvaUng buuons or o,h=, ,npa, d=v,„s 39 on 
taerface device M. can change the .ype of forecs required on object 34. 

„ „o change tn .he type of force is ctnrently .«,ui^ tn step i .0 then the process returns 
,o step ,06 to update the host application and return to step 1 .0 to ag.n check unf, s ch ch^ 

e y pe of force is required. When such a ch^g. is «quired. step ,12 ,s tntpientented, n 
host computer 12 detenrines an appropriate host commMd to send to nucroprocessor .6 Tn 
host contntands for host co™put« ,2 n«y each correspond to an assorted , . 
senll proce.,s impienKnted by microprocessor 26. For example, host contmands to prov 
ZLTforce, a spring force, a gravitational pull, a bumpy surface force, a vrtua, obst.c. 
o71 oth^r forces can be availab. to host computer ,2. These '--' co-ands can a. 
Llude a designation of the particular acutators 30 or degrees of freedom wh,ch are . ap^lu 
Lred force on object 34. The host commands can also mCude other command 
Irmation which might the force prodded b, a parncuiar J™^^^ 
damping constant. The host command may *o preferably ovemde *e lo^al --'^^^ 
procLor 26 and include lowlevd force values. A preferred command protocol and detatW 
d c jl of a set of host con^ands is descnbed ,n greater detail below with respect to Ftgur^s ^ 
a^ 4 In next step 1 14. the host computer sends .he host command to the mtcroproces..or 26 
Tver bus 24. The process then returns to step 106 to update ^ host applica.ion and to return to 
20 step 1 10 to check if another change in force is required. 

The local nucroprocessor 26 implement dte process branching from step 104 and s.ar.ing 
wid, step 116 in parallel wi,h fte hos, computer process described above. In step . 16 ^ 
interface device 14 is activated. For example, signals can be sen. between hos. computer 1. and 
in^rface device 14 .0 acknowledge that dte interface device is now active and can ^ ^ 
by hos. computer 12. From step 116. two processes branch .0 ind-cate dtat 
pLesses running simultaneously (multi-tasking) on local processor 26. In one processjtep 8 
is implemented, ,n wh.ch the processor 26 reads raw data from sensors 28. As descnbed ,n «ep 
88 of Figun: 4. processor 26 preferably reads posiuon data and no velocity or aceelerauon da-^ 
from sensors 28. In alternate embodimenus, .sensors 28 can tnclude velocity sensors and 
accelerometcrs for providing veloc.y and acceleration values of object 34. The sensor dau> read .n 
s,cp 1 18 can also tnclude other input, such as from an activated button or other conm,! of .nterface 
device 14. 

In next step 120. processor 26 processes the received raw data into .sensor data. As 
described in step 90 of Figure 4. thts processing preferably includes the two steps of computing 
velocity and acceleration data from the filtered posiuon data and f.ltering the velocity and 
acceleration data. Processor 26 can use its own local clock 21 to determine ihc tinung data needed 
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for computing velocity and acceleration. In addition, a hisloiy of previous recorded values, such 
as position or velocity values, can be used to calculate sensor data. In embodiments where velocity' 
and/or acceleration sensors are used, the calculation of velocity and/or acceleration is omitted. In 
next step 121. the processor 26 sends the processed sensor data to host computer 12 and also 
5 stores the data for computing forces, as described in the second branch process of processor 26. 
The process then returns to step 118 to read raw data. Steps 118, 120 and 121 are thu.s 
continuously implemented to provide current sensor data to processor 26 and host computer 12. 

The second branch from step 116 is concerned with an "actijator process" in which 
processor 26 controls the actuators 30 to provide forces to object 34. The second branch starts 

10 with step 122, in which processor 26 checks if a ho.st command has been received from host 
computer 12 over bus 24. If so. the process continues to step 124, where a force sensation 
process associated with the host command is selected. Such force sensation processes can be 
stored local to microprocessor 26 in, for example, memory 27 such as RA.M or ROM (or EPROM. 
EEPROM. etc.). Thus, the microprocessor might select a damping force sensation process if the 

15 high level command indicated that the damping force from this force sensation process should be 
applied to object 34. The available force sensation processes are preferably similar to those 
described above with reference to Figure 4, and may include algorithms, stored force profiles or 
values, conditions, etc. In .some embodiments, steps 118, 120, and 121 for reading sensor data 
can be incorporated in the force sensation processes for the microprocessor, so thai sensor data is 

20 only read once a force sensation process has been selected. Also, the host command may in some 
instances simply be a low-level force command that provides a force value to be sent to an actuator 
30, in which case a force sensation process need not be selected. 

After a force sensation process has been selected in step 124, or if a new host command 
has not been received in step 122, then step 126 is implemented, in which processor 26 determines 

25 a processor low-level force command (i.e. force value). The force value is derived from the force 
sensation process and any other data required by the force sensation process as well as command 
parameters included in relevant host commands, sensor data and/or timing data from local clock 
29. Thus, if no new high level command was received in step 122, then the microprocessor 26 
determines a force command according to the same force sensation process that it was previously 

30 using in step 126. In addition, the host command can include other command parameter 
information needed to determine a force command. For e.vamplc, the host command can indicate 
the direction of a force along a degree of freedom. 

In step 128, processor 26 outputs the determined processor force command to actuators 30 
to set the output force to the desired level. Before sending out the force command, processor 26 
35 can optionally convert the force command to an appropriate form usable by actuator 30, or actuator 
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interface 38 can perform such conversion. The process then returns to step 122 to check if another 
host command has been received from the host computer 12. 

Theactuaiorprocessof microprocessor 26 (steps 118, 120, 122, 124, 126, and 128) thus 
operates to provide forces on object 34 independently of host computer 12 according lo a selected 
5 force sensation process and other parameters. The force sensation process determines how the 
processor force command is to be determined based on the most recent sensor data read by 
microprocessor 26. Since a force sensation process indicates how forces should be applied 
depending on the position and other parameters of user object 34, the processor can issue low-level 
force commands, freeing the host computer to process the host application and determine only 
10 when a new type of force needs to be output. This greatly improves communication rales between 
host computer 12 and interface device 14. In addition, the host computer 12 preferably has the 
ability to override local control operation of microprocessor 26 and directly provide calculated or 
other force values as described above with reference to Figure 4. This override mode can also be 
implemented as a force sensation process. 

15 FIGURE 6 is a schematic diagram of an example of a gimbal mechanism 140 for providing 

two or more rotary degrees of freedom to object 34. Gimbal mechanism 140 can be coupled to 
interface device 14 or be provided with sensors 28 and actuators 30 separately from the other 
components of interface device 14. Gimbal mechanism 140 can be supported by a grounded 
surface 142, which can be a surface of the housing of interface device 14, for example 
20 (schematically shown as part of member 144). Gimbal mechanism 140 is preferably a five- 
member linkage that includes a ground member 144, extension members 146a and 146b, and 
central members 148a and 148b. Ground member 144 is coupled to ground 142. The members of 
gimbal mechanism 140 arc rotatably coupled to one another through the use of bcanngs or pivots, 
wherein extension member 146a is rotatably coupled to ground member 144 and can rotate about 
25 an axis A, central member 148a is rotaubly coupled to extension member 146a and can rotate about 
a floating axis D, extension member 146b is rotatably coupled to ground member 144 and can 
rotate about axis B. central member 148b is rotatably coupled to extension member 146b and can 
rotate about floating axis E, and central member I48a is rotatably coupled to central member 148b 
at a center point P at the intersection of axes D and E. The axes D and E are "noaling" in the sense 
30 that they are not fixed m one position as are axes A and B. Axes A and B are substantially 
mutually perpendicular. Gimbal mechanism 140 is thus formed as a five member closed chain. 
Each end of one member is coupled to the end of a another member. The f.vc-mcmbcr linkage is 
arranged such that extension member 146a. central member 148a, and central member 148b can be 
rotated about axis A in a first degree of freedom. The linkage is also arranged such that extension 
35 member 146b. central member 148b. and central member 148a can be rotated about axis B in a 
second degree of freedom. 
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User object 34 is a physical object that can be coupled lo a linear axis member 150, or 
linear axis member 150 can be considered part of object 34. Linear member 150 is coupled to 
central member 148a and cenu-al member 148b at the point of intersection P of axes D and E and 
extends out of the plane defined by axes D and E. Linear axis member 150 can be routed about 

5 axis A (and E) by rotating extension member 146a, central member I48a, and central member 148b 
in a first revolute degree of freedom, shown as arrow line 151. Member 150 can also be rotated 
about axis B (and D) by rotating extension member 50b and the two central members about axis B 
in a second revolute degree of freedom, shown by arrow line 152. In alternate embodiments, 
linear axis member can be linearly moved along floating axis C, providing a third degree of 

10 freedom as shown by arrows 153. In addition, linear axis member 150 in some embodiments can 
rotated about axis C, as indicated by arrow 155, to provide an additional degree of fiecdom. These 
additional degrees of freedom can also be provided with sensors and actuators. 

Sensors 28 and actuators 30 can be coupled to gimbal mechanism 140 at the link points 
between members of the apparatus and provide input to and output as described above. Sensors 
15 and actuators can be coupled to extension members 146a and 146b, for example. User object 34 is 
coupled to mechanism 140. User object 44 may be moved in both (or all three or four) degrees of 
freedom provided by gimbal mechanism'140 and linear axis member 150. As object 34 is moved 
about axis A, floating axis D varies its position, and as object 34 is moved about axis B, floaung 
axis E varies its position. 

20 FIGURE 7 is a perspective view of a specific embodiment of an apparatus 160 including 

gimbal mechanism 140 and other components of interface device 14 for providing mechanical input 
and output to host computer system 12. Apparatus 160 includes gimbal mechanism 140. sensors 
141 and actuators 143. User object 34 is shown in this embodiment as a joystick having a gnp 
ponion 162 and is coupled to ccnual member 148a. Apparatus 160 operates in substantially the 

25 same fashion as gimbal mechanism 140 described with reference to Figure 6. 

Gimbal mechanism 140 provides support for apparatus 160 on grounded surface 142, such 
ac 1 table top or similar surface. Gimbal mechanism 140 includes ground member 144, capstan 
drive mechanisms 164. extension members 1 46a and 1 46b, central drive member 148a, and cenu-al 
link member 148b. Ground member 144 includes a base member 166 and vertical support 

30 members 168 which are coupled to grounded surface 142. A vertical support member 168 is 
coupled to each outer surface of base member 166 such that vertical members 168 are in 
substantially 90-degree relation with each other. A capstan drive mechanism 164 is preferably 
coupled to each vertical member 168. Capstan drive mechanisms 164 i\rc included in gimbal 
mechanism 140 to provide mechanical advantage without introducing friction and backlash to the 

35 system. The drums 170 are rotated using a cable coupled to a pulley, which is driven by an 
actuator 143. 
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Extension member 146a is rigidly coupled to a capstan drum 170 and is rotated about axis 
A as capstan dr^im 170 is rotated. Likewise, extension member 146b is rigidly coupled to the other 
capstan drum 170 and can be rotated about axis B. Central drive member 148a is rotatably coupled 
to extension member 146a. and central l.nk member I48b is rotaiably coupled to an end of 
extension member 146b. Central drive member 148a and central link member 148b are rotatably 
coupled to each other at the center of rotation of the gimbal mechanism, which is the point of 
intersection P of axes A and B. Bearing 172 connects the two central members 148a and 148b 
together at the intersection point P. Gimbal mechanism 140 provides two degrees of freedom to an 
object 34 positioned at or near the center point P of rotation such that the object at or near point P 
can be rotated about axis A and B or have a combination of rotational movement about these axes. 
Also, object 34 can also be rotated or translated in other degrees of freedom, such as a linear 
degree of freedom along axis C or a rotary degree of freedom about axis C. 

Sensors 141 and actuators 143 are preferably coupled to gimbal mechanism 140 to provide 
input and output signals between apparatus 160 and microprocessor 26. In the described 

15 embodiment, sensors 141 and actuators 143 are combined in the same housing as grounded 
transducers 174. For example, transducers 174a and I74b are bi-diiectional transducers havmg 
optical encoder .sensors 141 and active DC scr>o motors 143. Passive actuators can also be used. 
The housing of each grounded transducer 174a is preferably coupled to a vertical support member 
168 and preferably includes both an actuator 143 for providing force in the first revolute degree of 

20 freedom about axis A and a sensor 14 1 for measuring the position of object 34 in the first degree of 
freedom about axis A. A rotational shaft of actuator 174a is coupled to a pulley of capstan drive 
mechanism 164 to transmit input and output along the first degree of freedom. Grounded 
transducer 174b preferably corresponds to grounded transducer 174a in function and operation. 

Object 34 is shown in Figure 7 as a joystick having a grip portion 126 for the user to grasp. 
25 A user can move the joystick about axes A and B; these movements are sensed by processor 26 
and host computer system 12. Forces can be applied preferably in the two degrees of freedom to 
simulate various haptic sensations. Optionally, other objects 34 can be coupled to gimbal 
mechanism 140. as described above. 

FIGURE 8 is a perspective view of a different embodiment of object 34 and supporting 
30 mechanism 1 80 that can be used in conjunction with interface device 1 4. Mcchamsm 1 80 includes 
a slotted yoke configuration for use with joystick controllers that is well-known to those skilled in 
the art. Mechanism 180 includes slotted yoke 182a. slotted yoke 182b, sensors 184a and 184b. 
bearings 186a and 186b, actuators 188a and 188b. and joystick 34. Slotted yoke 182a is rigidly 
coupled to shaft 189a that extends through and is rigidly coupled to sensor 184a ai one end of the 
35 yoke. Slotted yoke 182a is similarly coupled to shaft 189c and bearing 186a at the other end of the 
yoke. Slotted yoke 1 82a is rotatable about axis L and this movement is detected by sensor 184a. 
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In alternate embodiments, bearing 186a and be implemented as another sensor like sensor 184a. 
Similarly, slotted yoke 182b is rigidly coupled to shaft 189b and sensor 184b at one end and shaft 
189d and bearing 186b at the other end. Yoke 182b can rotated about axis M and this movement 
can be detected by sensor 184b. 

5 Object 34 is a joystick that is pivotally attached to ground surface 190 at one end 192 so 

that the other end 194 typically can move in four 90-degree directions above surface 190 in two 
degrees of freedom (and additional directions in other embodiments). Joystick 34 extends through 
slots 196 and 198 in yokes 182a and 182b, respectively. Thus, as joystick 34 is moved in any 
direction, yokes 182a and 182b follow the joystick and rotate about axes L and M. Sensors 184a- 

10 d detect this rotation and can thus track the motion of joystick 34, Actuators 188a and 188b allow 
the user to experience force feedback when handling joystick 34. Alternatively, other types of 
objects 34 can be used in place of or coupled to the joystick, and/or additional degrees of freedom 
can be provided to joystick 34. For example, the joystick can be provided with a rotary degree of 
freedom about axis K, as indicated by arrow 193. Sensors and/or actuators can also be included 

1 5 for such additional degrees of freedom. 

Other embodiments of mechanical interface apparatuses and transducers can also be used in 
interface device 14 to provide mechanical input/output for user object 34. For example, mechanical 
apparatuses which provide one or more linear degrees of freedom to user object 34 can be used. In 
addition, passive actuators having an amount of "play" can be provided. 

20 FIGURE 9 is a table 300 showing a number of preferred host commands that can be used 

in the embodiment of Figure 5, where host computer 12 sends high level host commands to local 
microprocessor 26, which implements locijl force sensation processes in accordance with the host 
commands. As discussed previously, low communication rates on bus 24 (Figure 1) can impede 
performance, specifically the accuracy and realism, of force feedback. The local microprocessor 

25 can implement force sensation processes based on host commands independently of the host 
computer, thus requiring less signals to be communicated over bus 24. Preferably, a 
communication language or force feedback protocol should ue standardized for the transfer of host 
commands from the host processor 16 to the local processor 26 to permit the efficient u-ansmission 
of high level supervisory commands (host commands) to local processor 26. 

30 A preferred embodiment contains two primary modes or "control paradigms" of operation 

for force feedback interface device 14: rate control and position control. These modes imply a 
classification scheme for host conamands parametrized by the conamand parameters. While the 
difference between rate control and position conu-ol is generally subtle to the user while he or she 
interacts with an application, the difference may be profound when representing force feedback 

35 information. Some of the commands can be used as either rate control or position conu-ol 
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commands. Other force feedback commands may be constructed in addition to. or as aliemaiivcs 
to, the following sample force feedback commands. 

Rate control refers to a user object mapping in which the displacement of the user object 34 
along one or more provided degrees of freedom is abstractly mapped to motion of a computer- 
simulated entity under control, such as an airplane, race car. or other simulated "player" or player- 
controlled graphical object. Rate control is an abstraction which makes force feedback less 
intuiUve because there i.s not a direct physical mapping between object motion and commanded 
motion of the simulated computer entity. In contrast, position control refers to a user object 
mapping in which displacement of the joystick handle or other user manipulable object directly 
dictates displacement of a simulated computer entity, so that the fundamental relation between 
joystick displacements and computer displacements is present. Thus, most rate control paradigms 
are fundamentally different from position control in that the user object (joystick) can be held 
steady at a given position but the simulated entity under control is in motion at a given commanded 
velocity, while the position control paradigm only allows the entity under control to be in motion if 
the user object is in motion. Position control host commands are described in greater detail below 
with respect to Figure 14. while rale control commands are described presently with reference to 
Figure 9. 

For example, a common form of rate control is a velocity derived abstraction in which 
displacement of the user object, such as a joystick handle, dictates a velocity of the simulated 
computer entity, such as a vehicle or other graphical object displayed on display screen 20. in a 
simulated environment. The greater the joystick handle is moved from the original position, the 
greater the velocity' of the controlled vehicle or player-controlled graphical object. Other common 
rate control paradigms used in computer games are acceleration controlled. Rate control force 
feedback commands roughly correspond to forces which would be exerted on a vehicle or other 
simulated entity controlled by the simulated environment through the force feedback interface 
device 14. Such forces are termed vehicle-centric forces. 

Herein, rate con';ol commands are divided into "conditions" and "overlays," although other 
classifications may be used in alternate embodiments. Conditions set up a basic physical model or 
background sensations about the user object including simulated stiffness, simulated damping, 
simulated inertias, deadbands where simulated forces diminish, and directional con.straints dictatmg 
the physical model's functionality. Multiple conditions may be specified in a single command to 
effectively superpose condition forces. Overlays, in contiast, are forces that may be applied in 
addition to the conditions in the background. Any number of overlays can preferably be provided 
in addition to condition forces. A condition can be specified by one condition command or by 
multiple condition commands. 
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Descriptions will now be provided for several types of forces 302, as referenced in table 
300. that can be implemented by microprocessor 26 from host commands. These forces include: 
restoring force, restoring spring, vector force, vibration, sluggish stick, wobble, unstable, button 
reflex joll, and ratchet force. The restoring force, restoring spring, sluggish stick, and unstable 
5 forces are considered condition forces. The vector force, vibration, wobble, and button reflex joll 
force.s are considered overlay forces. 

The forces 302 shown in table 300 can be implemented with host commands provided by 
host computer 12 to microprocessor 26. Examples 304 of host commands and their syntax arc 
shown in table 300 for each type of force 302. In the described embodiment, host commands 304 

10 preferably include a command portion 306 and a number of command parameters 308. Commands 
304 indicate the t>pe of force which the host computer 12 is instructing the processor 26 to 
implement. This command portion may have a corresponding force sensation process which the 
processor 26 can retrieve from memory 27 and implement Command parameters 304 arc values 
or indicators provided by the host computer 12 which customize and/or modify the type of force 

15 indicated by command portion 304. For the following preferred rate control embodiments, most of 
the command parameters control different forces in the same way. The magnitude parameter is a 
percentage of a maximum magnitude corresponding to a maximum force able to be output by 
actuators 30. The duration parameter usually corresponds to a time interval for applying the 
particular force model, or can be indefmitely applied. The style parameter may select a direction in 

20 which to apply the force model, and/or a degree of freedom along which to apply the force model. 
Although not listed in Figure 9, all of the described types of forces 302 can have addiuonal 
parameters or incorporate other properties into the listed parameters. A "deadband" parameter 
could specify a size of a region where a force would be small or zero. A parameter can be included 
indicating whether a force is bi-directional or uni-directional along a degree of freedom. Subclass 

25 310 indicates a classification of the types of forces 302 as either conditions or overiays. as 
explained above. 

FIGURES lOa-c are graphs iliusuating force versus displacement profiles for a restoring 
force. The force in graph 312 of Figure 10a is bi-direciional, where the force on the right side of 
the vertical axis is applied in one direction along a degree of freedom, and the force on the left side 
30 of the vertical axis is applied in the opposite direction along that degree of freedom. The force 
shown in graph 314 of Figure 10b is uni-directional. Preferably, whether the force is uni- 
directional or bi-directional is specified with, for example, the style parameter 308 of the command 
306 shown in table 300 of Figure 8. In addition, the de.sired degrees of freedom along which the 
restoring force is to be applied are also preferably specified in the style parameter. 

35 A restoring force applied to user object 34 always points back towards an origin position O 

(or "neutral position") of the user object along a degree of freedom. For example, the origin 
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poshion for a jovsUck «n be Joysuck. cemcr posi.icn, as sKo»n in F,6.r.s 7 a^d 8^ 

of e .onng fo.., specified by ,h= n„3gn„ude co™d p.a„..er ^m™ 
"C ineite direct to, .he range 316 along .he degree of freedom of <^ user ot, c,. 
Zlm force .agnrude F is preferably -red ,o abour of d« rnaxi,™.,. poss, le ™.p^ 
Te" a ,he seleaed degree of freedom, so ,ba, jolu and vibrarions can be ovcrlad on .op o. m= 
ZL sen.a.ion (descrrbed below). As *e objec. is n»ved roward ,he ongrn p»,..on O^^ 
"„Z orce .s co„s.an, un.il .he user ohjec. is moved »i,hin a locali^d r=g,on R abou. .he or,g,n 
: on When .he user objec, is in .he locah^d ..gion R, *e applied force rapidly drops .o «™ 
o sll value Thus, .he resroring force pn,f,le provides a cons,an. ■■res.onog s=ns3.,on to. 
i:::^ l .» .^e orig^ pos^ion when me obiee. U in range 316. Th,s res,or.ng 

forces ,hen d,nun,shes or vanishes as .he objec. .ears and reaches .he or,g,n pos,.,on. 

,„ Hgure IOC. U,e resroring force is shown sim,iarly .o .he force in Figure 10a, cxcep. U«, 
U,e applied force is abou. zero in an extended reg.on 3 1 S, abou, .he origin P°»- ^'^^ ' ' 
known as a "deadband". and allows .he user .o have son. freedom .o move objec. 34 for a shor^ 
dLce around ,he orig,n before forces are applied. A res.or.ng force sensa,,on can be very ably 
ai^rd .ra rate conuoU'S- 'o .he si.ua,io„ of hi..i„g a wall or some oU,er obsm.c.,on wh„e 
controlling a simulated vehicle. 

FIGURES 1 la-llc are graphs illus.ra.ing force versus displacemem profiles for a restoring 
spring force. Rather tan m^ouining a constant magnitude over much of its positive or negauve 
■IsplLmen.. as provided by Ute res.oring force of F.gures lOa-lOc. a restormg spnng fo « ar^ 
linLy over an appreciable portion of .1. user objec's displaccntent. and ,s pro^ r uon^ »^ 
object 34's dtstance from the or.gtn position O. A restoring spring force appl.d ' "^^^ 
always points bac. towards the neutra, position along a degree of freedom. Graph .20 of F.gur 
1 la shows the bi-direcional case, and graph 322 of Hgure 1 lb shows the uni-direcfonal ca.se. A 
; deadband specfied by a deadband parameter ,s provided about the origin po^tton ass^v«, n 
graph 324 of Figure 1 Ic. The restoring spring force can have a sprtng eocfnctem parameKr <o 
describe a desired "sriffness" of ttie object 34. 

The sluggish foree er«..es a damping force on user object 34 having a magn.tude 
proporiional to the velocity of the user objec, when moved by .he user. An example of .hts .ype o 
,0 damping force is described above wi,h .espec. to s.ep 82 of Figure 4. The degree of vtscost.y o 
the sluggish force can be sp«:if.ed by a viscous damping coefftcent, wh.ch can be "P-^^"; ' 
percentrge of a maximum damping coefficient. The sluggtsh stick force is particularly sutted for 
rate control applications to simulate controlling, for example, a very heavy vehrcle th^ .s poorly 
responsive to the movemem of the user object. The unstable force creates an inveried pendulum 
sole instability. Alternatively, the unstable force is modeled on a spring having a negauve spring 
constam (an unsuble or diverging spring). A force is applied to .he user objec, in a direchon awa, 
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from the object's origin position and is increased as the user object is moved further away from the 
origin position. This creates a force that makes it difficult for the user to bring the object to the 
origin position. This force can be used as another vehicle-related sensation, and could replace a 
restoring spring force when, for example, a simulated vehicle guidance control is damaged. 

In alternative embodiments, the condition forces described above can be commanded using 
only one generic host command with a number of parameters to control the characteristics of the 
condition forces. 

The condition commands can be provided in the background while overlay commands are 
applied in addition to or "over" the condition forces. For example, a sluggish damping force can 
be provided as a background force to the user object, and a "jolt" overlay force can be commanded 
over the sluggish force to provide a quick, jerky motion on the user object for a few seconds. Of 
course, overlay forces may also be applied exclusively when no other forces are being applied, or 
may cancel other previously-commanded forces if desired. The example overlay forces shown in 
Figure 9 are described below. 

HGURE 12 is a graph 326 illustraung a vector force model. A vector force is an overlay 
command, and thus can be applied in addition to the condition forces described above. It is a 
general force applied to the joystick in a given direction specified by a direction command 
parameter. Figure 12 shows a two-dimensional representation of the vector force in an example 
direction in the X- Y plane of a user object having two degrees of freedom. 

FIGURES 13a- 13b are graphs illustrating force versus time profiles for a vibration force. 
Figure 13a is a graph 328 showing a bi-directional vibration force while Figure I3b is a graph 330 
showing a uni-directional vibration force. The vibration command shown in Figure 9 accepts 
magnitude, frequency, style, direction, and duration command parameters. The frequency 
parameter can be implemented as a percentage of a maximum frequency and is inversely 
proportional to a time interval of one period. Tp. The direction command parameter can be 
specified as an angle or degree of freedom. The .style parameter can indicate whether the vibration 
force is uni-direcuonal or bi-directional. In addition, a duty cycle parameter can be provided in 
alternate embodiments indicaung the percentage of a time period that the vibration force is applied. 
Also, a command parameter can be included to designate the "shape" or profile of the vibration 
waveform in the time axis, where one of a predetermined number of shapes can be selected. For 
example, the force might be specified as a sinusoidal force, a sawtooth -shaped force, a square 
waveform force, etc. 

A wobble force paradigm is another overlay force that can be commanded by host computer 
12. This force creates a random (or seemingly random to the user), off-balance force sensation on 
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thc user object. For exan^ple, it car, simulate an erratic control for a damaged veh.cle. A style 
parameter rrught also specify a type of wobble force from a predetermined Ust of different types. 

The jolt force is typically a short, high magnitude force that is output on the user object, 
and can be u.sed. for e.xample. to notify the user of an event or simulated object in the computer 
5 environment. The jolt force can be used as an overlay force, which can be feh in addition to any 
condition forces in effect. Typical parameters include the magnitude of the force of the jol , the 
duration of the jolt, and direction(s) ordegree(s) of freedom in which the jolt is applied, which can 
be specified as an angle or particular degrees of freedom. 

The button force is not an actual force but may be used as a command to trigger other 
10 forces when an mput device 39 is activated by the user. In many game situations, for example, it 
n,ay be advantageous to tngger a force as a direct response to pressing a button or other input 
device 39 on the interface apparatus 14 rather than generating the force from a host command after 
processing the pressed button on the host computer 12. 

For example, a common force to use in conjunction with a button command is the jolt 
15 force A specific command, e.g.. BUTTONJOLT. can be provided to cause a jolt force 
whenever a specified button is pressed, and which includes button and jolt command parameters. 
V^Tien the button jolt command is received by microprocessor 26, the microprocessor can run a 
button check as a background process until commanded to terminate the button background 
process. Thus, when the microprocessor 26 determines that the u.ser has pressed a button from the 
20 sensor data, the jolt force can be overlaid on any existing forces that are output. 

The button command sets up the microprocessor 26 to output a force when the other input 
device 39 has been activated. The button command may accept a number of command parameters 
including, for example, button and autofire frequency parameters (in addition to any command 
parameters specific to the desired force to be output when the button is pressed). The button 
25 parameter selects the particular button(s) which the microprocessor 26 will check to be acuvated by 
the user and which will provide the desired forces. For example, a joy.suck may have muluple 
buttons, and the software developer may want to provide a force only when a particular one of 
those buttons is pressed. A duration parameter can detemiine how long the jolt lasts after the 
button is pressed. Tlie "autofire" frequency parameter designates the frequency of a repeaung 
30 force when the user holds down a button. For example, if the user holds down a particular button, 
the microprocessor can automaucally repeat a jolt force after a predetermined time interval has 
passed after the user first pressed the button. The autofire parameter can also optionally designate 
whether the autofire feature is being used for a particular button and the desired time interv-al before 
the repeating forces are applied. 
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FIGURE 14 is a table 332 showing a number of preferred positiori control host conimands 
that can be used in the embodiment of Figure 5. Herein, -posiuon control- refers to a mapping of 
a user object in which displacement of the joystick handle or other user object directly dictates 
displacement of a computer-simulated entity or object. The mapping can have on arbitrary scale 
5 factor or even be non-linear, but the fundamental relation between user object displacements and 
computer object or entity displacements should be present. Under a position control mapping. ±c 
computer-controlled entity does not move unless the user object is in motion; a static user object 
dictates static commands to microprocessor 26 from host computer 12. 

Position control is not a popular mapping for traditional computer games, but is effectively 
10 used tn other applications such as the graphical user interface (GUI) embodiments disclosed 
herem Position control is an intuitive and effective metaphor for force feedback mteracfons 
because it is a direct physical mapping rather than an abstract control paradigm. In other words 
because the user object experiences the same physical manipulations as the entit>' being controlled 
within the computer, position control allows physical computer simulations to be directly reflected 
15 as realistic force feedback sensations. Examples of position control in computer environments 
might be controlling a paddle in a pong-style tennis game or controlling a cursor m a GUI or 
windows desktop environment. Contrasted with rate control's vehicle-ecnlric forces, position 
control force feedback roughly corresponds to forces which would be perceived directly by the 
user. These are "user-ceniric" forces. 

20 Descriptions will now be provided for several types of position control forces 334, as 

referenced in table 332. that can be implemented by microproces.sor 26 from host commands. 
These forces include: vector, groove, divot, texture, barrier, f.eld, paddle, and button reflex jolt. 
Many of the examples 336 of host commands corresponding to these forces use magnitude and 
style parameters as discussed with reference to the rate control paradigms. As with the rate control 

25 commands, command parameters of the same name generally have the same properties for different 
host command.s. However, the duration parameter is typically not used for position control 
commands as much as for rate control commands, since the duration of the position control forces 
are typically applied depending on the current position of the user object. The position control 
force models thus generally remain in effect until the host computer 12 issues a new host force 

30 command or a clear command. In alternate embodimenls. a duration parameter can be used. 

Preferred parametrizations for described position control conmands arc summarized in 
Figure 14. All the forces listed below can include additional command parameters, such as 
deadband parameters, or incorporate other properties into the parameters li.sted in Figure 14. 
Similar to the host commands shown in Figure 9. host commands 336 preferably include a 
35 command portion 338 and a number of command parameters 340. Commands 336 indicate the 
type of force which the host computer 12 is instructing the processor 26 to implement. This 
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command portion may have a corresponding force scnsauon process which the processor 26 can 
IZ. uZ rr^emory 27 and implement. Command ponions 33S can be speafted m v.nually an> 
form. 

Avcc,o,forcc,.ag=„cnl fo:c. having . magnUude ^ direcdon. " R^"'' ' ; 

a polar repres=n«.ion of toe ««or force. Mos. position =on,™l «nsa«ons wUl be genenncd by he 
p:;aJcrMeve,oper .sing a vecior force co„,n,and and appropria„ — - 
Lgranuning consmics. A duraUon p3«me«r is typically no. needed stnce the host 1. 
microproces^r 26 can terminate or modify the force based on user object motions, not t.me. 

nOURE ■ 5 is a graph 342 showing a force versus displacement relationship for a groove 
force of the p-esem tnventton. groove force prov.des a linear d«en. sensadon along a ^vej^ 
egtee of freedom, shown by ramps 344, The user object feels like tt ts captured ,n a groove 
wL there is a restoring force along the degtee of f«edom to keep the suck ,n the groove Tin 
Te^olg force groove ts centered about a center groove posidon C located a. the cut^nt locatton 
rrobjec. When the host command was received. MemaUvely, *e ^2:Z^ o< 
groove position can be specifed from a command parameter along one or -^ fi'^ l 
freedom Thus, if the use, attempts to move the user object out of the groove, a reststtng force 

applied. 

The magnitude (stiffness) parameter specifies the amount of force or resistance applied. 
Optionally, a "snap-out" feature can be implemented withm the groove force sensauon process 
where the groove forces turn off when d^e user object deviates from the groove by a g.ven snap, 
out distance, shown as distance S. Thus, the microprocessor 26 would receive a groove command 
having a snao distance magnitude. When the microprocessor detects the user object moving 
outside this snap distance, it turns off the groove forces. This snap-out feature can be .mplemented 
equally well by the host computer 12 sending a clear command to tun. off forces Also 

; Idband DB can also be provided to allow the user object to move freely near the center groove 
position C. specified with a deadband command parameter. A style command parameter indicate^ 
L orientation of the groove along one or more degrees of freedom (. g-. horizontal vertical 
diagonal). For example, horizontal and vertical grooves can be useful to provide forces for scroll 
bars in windows. A user moving a cursor in a graphical user interface can feel groove forces 

0 moving the cur.sor and user object toward the middle of the scroll bar. The deadband g.ves the 
user room to move the cursor within the scroll bar region. The snap-out distance can -be used 
free the cursor/user object from forces once the cursor is moved out of the scroll bar region. 

A divot IS essentially two (or more) orthogonal grooves that provide restoring forces in 
more than one degree of freedom. This provides the sensation of a point detent along a given 
55 degree of freedom. If the divot is provided in two degrees of freedom, for example, then the user 
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Object feels as it if has been captured in a circular depression. The user object is captured at a point 
where there is a restoring force along both axes to keep the user object at the point. The snap-out 
feature of the groove force can also be implemented for the divot. In addition, the dcadband 
feature of the groove can be provided for the divot conunand. 

5 A texture force simulates a surface property, as described above with reference to Figure 4. 

A texture is a spatially varying force (as opposed to vibration, a time varying force) that simulates 
the force felt, for example, when a stick is moved over a grating. Other types of textures can also 
be simulated The user object has to be moved to feel the texture forces, i.e.. each 'bump" of the 
grating has a specific position in the degree of freedom. The texture force has several 
10 characteristics that can be specified by a programmer/developer using the host commaitd and 
command parameters. These command parameters preferably include a magnitude, a gru. and a 
style The magnitude specifies the amount of force applied to the user object at each "bump" of the 
grating The grit is basically the spacing between each of the grating bumps. The style command 
parameter can specify an orientation of the texture. For example, the style can specify a honz.omal 
15 grating, a vertical grating, or a diagonal grating (or a superposition of these gratings). 
Furthermore, the style parameter can specify if the texture is felt bi-directionally or uni-d,rectionally 
along a degree of freedom. Alternatively, addiuonal command parameters can be provided to 
control the position of the "bumps" of the texture force. For example, information can be included 
to instruct the distance between bumps to var>' exponentially over a distance, or vary according to a 
^0 specified formula. Alternatively, the texture spacing could vary randomly. In yet other 
embodiments, the command parameters can specify one of several available standard tcxtme 
patterns that microprocessor 26 can retrieve from memory. 

A baiTier force, when commanded, simulates a wall or other obsiruciion placed at a location 
in user object space, and is described above with reference to Figure 4. The host command can 
25 specify the hardness of the barrier (magnitude of the force applied), the location of the barrier along 
the degree of freedom, and the snap distance or thickness of the barrier. A barrier can also be 
provided with a compliance or springiness using a spring constant. Horizontal bamcrs and vertical 
barriers can be provided as separate host commands, if desired. As indicated in graph 346 of 
FIGURE 16. a barrier force only has a finite thickness. The force increases .steeply as the user 
30 object is moved closer into the barrier (past poinl B). The snap-through distance defines the size of 
the region where the barrier is felt by the user. If the user object is moved into a barrier, and then 
is moved past the thickness of the barrier, the barrier force is turned off The barrier force can act 
as a hard obstruction, where the microprocessor provides maximum force magnitude to the user 
object 34. or as a bump or softer barrier, where a smaller force magnitude is applied (as specified 
35 by the magnitude command parameter). The bairier can remain for an extended period unless 
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removed or moved lo a new location. Multiple barriers can also be provided ir, succession along a 
degree of freedom. 

Alternatively, the barr,er force can be provided by sending a host command having only- 
two command parameters, hardness and location. The hardness parameter can specify the hetght 
and slope of the reststive force. As shown in graph 348 of Figure 16. the user object can move 
from left to right along the distance axis. The user object feels a resistive force when httung the 
barrier at point B. After the user object has been moved to point S (the snap-distance), the force ts 
applied to the user object in the opposite direction (a negative magnitude force), which decreases as 
the user object is moved in the same direction. This simulates a bump or hill, where the force .s 
resistive until the user object is moved to the .op of the bump, where the force becomes an 
assisting force as the object is moved down the other side of the bump. 

A force field type force attracts or repulses the user object with respect to a specific 
position This force can be defined by command parameters such as a field magnitude and the 
specific field origin position which the force field is applied with respect to. A sense parameter 
bTmcluded to indicate an attracdve field or a repulsive field. For example, the force field can be an 
attractive field to simulate a force of gravity between the field origin position and a user-controlled 
cursor or graphical object. Although the field origin position can be thought of as a gravttauonal 
mass or an electric charge, the attractive force need not depend on the inverse square of 
displacement from the specific posiuon; for example, the force can depend on an inverse of the 
displacement. The attractive force field also anempts to maintain the user object at the field origin 
position once the user object has been moved to that position. A repulsive field operates similarly 
except forces the user object away from a specified field ongin position. In addition, ranges can be 
specified as additional command parameters to limit the effect of a force field to a panicular 
distance range about the field origin position. 

FIGURES 17a-17i ai^ diagrammatic illustrations of a "paddle" computer object 350 
interacting with a "ball" computer object or similar object 352. The.sc computer objects can be 
displayp'^. on display screen 20 by host computer 16. The force interactions between the ball and 
paddle can be controlled by a software developer using a host coimnand. as explained below. In 
the described example, paddle object 350 is controlled by a player by a position control paradigm 
) such that the movement of paddle object 350 is directly mapped to movemem of user object 34. In 
alternate embodiments, ball object 352 or both objects can be controlled by players. 

Figures 17a-17h show how paddle object 350 interacts with a moving ball object 352 as 
ball object 352 collides with the paddle object. In Figure I7a, bail 352 first impacts paddle 350. 
Preferably, an initial force is applied to user object 34 in the appropriate direction. In Figures 17b 
5 and I7c, ball 352 is moving into the compliant paddle or "sling". Preferably, a force based on a 
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Simulated mass of ball 352 is felt by the user through user object 34 which is appropriate to the 
simulated velocity of the ball (and or the paddle), the simulated compliance of the paddle (and/or 
the ball), and the strength and direction of simulated gravity. These factors (and other desired 
physical factors) can preferably be set using a host command with the appropriate parameters. For 
5 example, the following host command can be used: 

PADDLE (B_mass. B_veLx. B_vel_y. Gravity, Sense. Compliance.X, Compliance.Y, style) 

where the command parameter B_mass indicates the simulated mass of the ball, B_ve!_x and 
B_vel_y are the velocity of the ball, gravity is the strength of gravity, sense is the direcuon of 
gravity, and Compliance.X and Compliance.Y are the simulated compUance or stiffness of the 

10 paddle object 34. Other parameters can also be included to conu-ol other physical aspects of the 
computer environment and interaction of objects. For example, a simulated mass of the paddle can 
also be specified. Also, the ball 352 can be displayed as a compressed object when it impacts 
paddle 350, with, for example, a reduced height and an oval shape. In addition, damping 
parameters in the x and y axes can also be included in the paddle command to add a damping force 

1 5 to the collision between the ball and paddle in addition to the compliance (spring) force. Also, the 
parameters such as the compUance and/or damping of the paddle might be allowed to be adjusted 
by the user with other input 39 or a third degree of freedom of user object 34. The style parameter 
of the paddle command might select one of several different predetermined paddle configurations 
that are available and stored in. for example, memory 27. The configurations can have different 

20 paddle lengths, widths, compliances, or other displayed and/or force characteristics of a paddle. 

In Figure 17d. the ball has reached a maximum flexibUity point of paddle 34 and can no 
longer move in the same direction. As shown m Figures 17e through 17g, the ball is forced m tlic 
opposite direction due to the compliance of the paddle. In addition, the user may preferably exert 
force on user object 34 to direct the ball in a certain direction and to add more velocit>' to the ball's 

25 movement. This allows the user a fine degree of conUoI and allows a significant application of 
skill in directing the ball in a desired direction. The force feedback paddle is thus an improved 
component of -pong' type and other similar video games. In addition, the paddle 350 can 
optionally Hex in the opposite direction as shown in Figure 17h. An interface apparatus providing 
two linear (X and Y) degrees of freedom to user object 34 as well as a rotating ("spin") third 

30 degree of freedom about the Z axis (or C axis is Figure 6) is quite suitable for the paddle-ball 
implementation. 

A schematic model of the forces interacting between ball 352 and paddle 3.50 is shown in 
Figure 17i. A spring force indicated by spring constant K is provided m both degrees of freedom 
X and Y to indicate the springiness of the paddle 350; g is a gravity direction. In addition, a 
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damping force indicated by damping constant B is also provided to slow the baJl 352 down once it 
contacts paddle 350. The spnng and damping forces can also be applied in one degree of freedom 

The paddle control algorithm is a dynamic algorithm in which microprocessor 26 computes 
interaction forces while a ball compresses the paddle and then releases from the paddle. The 
paddle command is sent by host computer 12 when the ball contacts the paddle. The paddle 
command reports ball IccaUon to the host computer so that the host can update graphics displayed 
on display screen 20 during the interaction period. In presently preferred embodiments, the 
updates only need to be provided at about 60 Hz to the host, since most displays 20 car only 
display at that rate. However, the forces should be computed and output at about 500 Hz or more 
to provide a realistic "feel" to the interaction. Thus the local microprocessor can compute the 
forces quickly while occasionally reporting the sensor readings of the paddle to the host at a slower 
rate Other types of video game or simulauon interactions can also be commanded with a high- 
level host command in a similar fashion. In addition, in alternative embodiments, host computer 
P can control the acwators 30 directly to implement the paddle and ball force feedback, without 
sending any high level host commands. Also, similar paddle-ball interactions and forces can be 
provided between interactions of other types of graphical objects, such as between a cursor .n a 
GUI and another object or surface. 

FIGURE 17J is a diagrammatic illustration of a 2-D implemeniation of displayed graphical 
objects on display screen 20 which can be implemented with the paddle host command descnbed 
above or implemented in a GUI or other graphical environment. A playing field is displayed m 
which action is to take place, and two goals 368 and 370 are provided on the playing field. Two 
paddles 360 and 362 are displayed which are moved around the playing field. Paddles 360 and 
362 are shown as vertically-aligned .segments having a length and a relatively small width, but can 
be oriented and/or shaped quite differenUy in other embodiments. For example, other geometric 
shapes, images of tennis rackets, or images of a person holding a tennis racket can be used in place 
of the paddles. Paddles 202 and 204, ball 206, goals 201 and 203. and any other computer- 
generated objects that are included in the simulauon are genencally referred to herein as "graphical 
objects." 

In one embodiment, paddle 360 can be controlled by host computer system 12. and p:.ddle 
362 can be controlled by the user by physically manipulating the user object. Ball 352 can be 
moved on display .screen 20 according to simulated physical parameters, such as velocity, 
acceleration, gravity, compliance of objects, and other parameters as discussed previously. When 
the ball 352 collides with paddle 362, the paddle flexes, and the user feels the collision force. For 
example, if ball 352 is moving in direction 364. then the user feels a force in the equivalent degrees 
of freedom of user object 34. In some embodiments, both the paddle 362 and the ball 364 can be 
moved in direcuon 364 to simulate the paddle being pushed back by the ball. FIGURE 17k shows 
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a similar embodiment in which a first-person perspective view (or simulated 3-D v.ew) of the 
graphical objects is shown on display screen 20, where ball 352 is a sphere. 

The user can also move the user object so that the paddle moves in a direction 366. The 
user will thus feel like he or she is "carrying" the weight of the ball, as in a sling. The ball will 
then be released from the paddle and move toward the other paddle 360. As is well known, a goal 
in such a game might be to direct the ball into the opposing goal. Thus, the user can ir>' to direct 
the ball into goal 368, and the host computer can control paddle 360 to direct the ball into goal 370. 
Paddles 360 and 362 are used to block the ball from moving into the defended goal and to direct 
the ball back at the desired goal. By moving the paddle in a combination of direction 366 and up 
and down movement, the user can influence the movement of the baU to a fine degree, thus 
allowing a player's skill to influence game results to a greater degree than in previous games 
without force feedback. In addition, other features can be included to funher influence the ball s 
direction and the forces felt by the user. For example, the orientation of the paddle can be changed 
by rotating the paddle about a center poim of the paddle. This rotation might be sensed from a 
"spin" degree of freedom of the user object about an axis C. as described above with reference to 
Figures 6 and 7. Force feedback could thus be appropriately applied in thai spin degree of 
freedom. Other features can also be provided, such as allowing a ball to "stick" or be trapped to a 
paddle when the two objects collide and/or when a button is pressed by the user. The user could 
then activate or release the button, for example, to release the ball at a desired time. 

In another embodiment, paddle 360 can be controlled by another u.ser rather than host 
computer 12. For example, a second interface device 14 can be connected to another input/output 
port of host computer 12 and can be used to control paddle 360 by a second u.ser. Each player 
would therefore feel the forces on their respective paddle from the ball directed by the other player. 
In addition, if the two paddles 360 and 362 were brought into contact with one another, each 
player could feel the direct force of the other player on each player's user object. That is, a first 
user's force on his user object would cause his paddle 362 to move into the other paddle 360. 
which would cause both the first and second users to fee! the collision force. If the first paddle 
362 were allowed to push the other paddle 360 across the screen, then the second user would feel 
the first user s pushing force. The first user would feel similar forces from the second user. This 
creates the effect as if each player were pushing the other player directly. Such pushing or "tug of 
war" games between two users can take several different embodiments. 

Furthermore, the second interface device 14 need not be connected to computer 12. 
Instead, host computer 12 can be coupled to a second host computer through a direct or network 
interface, as is well to those skilled in the art. The movement of a first user object would thus be 
communicated from the first host computer to the second host computer, which would then 
command forces on the second user object, and vice-versa The embodiment of Figure 17k is 



PCT/1B96/0144I 

WO 97/21160 

-43- 

appropriate for such an embodtmeni, where each user can v.ew paddle 362 as the paddle under h.s 
own control on h.s own display screen 20 and paddle 360 as the other player's paddle. 

This concludes the descripuon of position control paradignis. 

In addition, a clear command is pi^ferably available ,o the host computer. This command 
can include a parameter specifying paiticular degrees of freedom and allows the host computer to 
cancel aU forces in the specified degrees of freedom. This allows forces to be removed before 
other forces are applied if the programmer docs not wish to superimpose the forces. Also, a 
configuration host command can be provided to .mttally set up the interface device 14 to recetve 
particular communication parameters .md to specify which input and output will be used for a 
particular application, e.g. the ho.st computer can instruct local microprocessor 26 to report .specific 
information to the host computer and how often to report the information. For example, host 
computer 12 can instruct microprocessor 26 to report position values from particular degrees of 
freedom, button states from particular buttons of interface device 14. and to what degree to report 
errors that occur to the host computer. A "request information" command can also be sent by host 
> computer 12 to interface device 14 to receive infomtation stored on the interface device 1 4 at the 
li„,e of manufacture, such as serial number, model number, style information, calibrauon 
parameters and information, resolution of sensor data, resolution of force control, range of motion 
along provided degrees of freedom, vendor identification, device class, and power management 
information, etc. 

0 In addition the above described forces can be superimposed. The host computer can send 

a new host command while a previous host command is still in effect. This allows forces applied 
to the user object to be combined from different controlling commands. The microprocessor 26 or 
host computer may prevent cenain commands that have contradictory effects irom being 
superimposed (such as a restoring force and a restoring spring). For example, the latest host 
05 commandsentcanoverridepreviouscommandsifthosepreviouscommandsconflict with the new 
command. Or, the conflicting commands can be assigned priorities and the command with the 
highest priority overrides the other conflicting commands. 

It should be noted that the high-level host commands and command parameters described 
above are merely examples for implementing the forces of the pre.sem invention. For example. 
30 command parameters that are described separately can be combined into single parameters having 
different ponions. Also, the distinct commands shown can be combined or separated m different 
ways, as shown above with the example of the condition command for providing multiple rate 
control condition forces. 
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In addition to conunon interface devices with one or two rectangular or spherical degrees of 
freedom, such as standard joysticks, other interface devices can be provided with three or more 
degrees of freedom. When the third degree of freedom is about an axis along the stick itself, those 
skilled in the art call it "spin" or "twist." Each degree of freedom of a user object can have its own 

5 dedicated high-level host command. By independently associating high-level ho.si commands to 
each degree of freedom, many possible combinations of position control, rate control, and other 
abstract mappings can be implemented with interface devices. Multiple control paradigms may also 
be mixed in a single degree of freedom. For example, a joystick may have position control for 
small deviations from the origin in a degree of freedom and rate control for large deviations from 

10 the origin in the same degree of freedom. Such a mixed control paradigm can be referred to as a 
local position/global rate control paradigm. 

FIGURE 18 is a diagrammatic illustration of display screen 20 displaying a graphical user 
interface (GUI) 500 used for interfacing with an operating system implemented by computer system 
12. A preferred embodiment of the present invention implements force feedback technologies to 

1 5 embellish a graphical user interface with physical sensations. By communicating with force feedback 
interface device 14 or a similar apparatus that provides force feedback to the user, the computer 1 2 can 
present not only visual and auditory information to the user, but also physical forces. The.se physical 
forces can be carefully designed to enhance manual performance by, for example, reducing the 
difficulty of required "targeting" tasks. Such force feedback sensations can be used to faciUtate 

20 interaction with computer operating systems for all users. In addition, those users who suffer from 
spastic hand motion and other dexterity-debilitating conditions reap great reward from the addition of 
these force feedback sensations. 

The addition of computer generated force feedback sensations to a windows operating system 
environment can enhance manual performance in at least two ways. First, physical forces can be used 
25 to provide haplic sensory cues on user object 34 which increase a users perceptual understanding of 
the GUI spatial "landscape" portrayed on display screen 20. For example, sensations of physical 
bumps or textures which are applied to user object 34 as the user moves a cursor across the screen can 
be used to indicate to the user that he has positioned the cursor within a given region or crossed a 
particular boundary. 

30 Second, computer-generated forces can be used to provide physical constraints or assistive 

biases which actually help the user acquire and maintain the cursor at a given target displayed on 
screen 20 within GUI 500. For example, an attractive force field can be used to physically attract user 
object 34. and thus the cursor controlled by user object 34. to the location a.ssociated with a given 
target such as an icon. Using such an attractive field, a user simply needs to move a cursor on the 

35 screen close to the desired target, and the force feedback interface device 14 will assist the user in 
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nK)ving the cursor to the target. Many other abstract force feedback sensations can be used to enhance 
and embellish the wide variety of GUI-based metaphors. 

Herem the manual tasks of the user to move a cursor displayed on screen 20 by physically 

.an.puiat.,u;er object 34(alsorefe.edtoasa>y.icalob3e^ 
5 ades.redlocauonordisplayedobject,aredescribedas"targetmg acuvmes. ^^^^^ l"^^^^^^ 

herein are defined regions in the GUI 500 to which a cursor may be moved by he user that a. 

aslcilted withoneor more forces and which are typically associated with ^^^^^^^^^ 

500 such targets can be associated with, for example, grapbca] objects such as .cons, pull-do^n 

nlnuLs ^d buttons. A target usually is defined as the exact dimensions of us assoc. ated 
.0 robiect. and is superimposed and "attached" to ,ts associated graphical object such .a. ^ 

arget has a constant spatial position with respect to the graph.ca, object (,.e when he ,r.^^^ 

Zct is moved, its target also moves the same distance and d.rection). Usually, "graphical objects 

:Tt::::age;appelgonthe.splayscreen* 

an operating system funct.on. such as displaying .mages. execut.ng an appl.cat on program^ - 
,5 perfonmng another computer function. For simplicity, the term "targe r may - ~ 
graphic^ object with wh^ch the target .s associated. Thus, an icon or wmdow .tself s of^n .fer« 
o hlein as a "target". However, more generally, a target need not "^^^^^^^ 
graph.c^ object associated w.th the target. For example, a target can ^ defined s " 
displayed area of an associated graphical object, or the target can be defined as only ^ ^^^^^^^ 
20 graphical object. T.e target can also be a different size and/or shape than us --^^^^^^^^^^ 
object, and may even be positioned a distance away from its associated graph.cal obje t. The enu. 
screen or background of GUI 500 can also be considered a "target" which may prov.de forces on user 
object 34 In addition, a single graphical object can have multiple targets associated w.th .t For 
example, a window might have one target associated with its entire area, and a separate target 
25 associated with the title bar or comer button of the window. 

Upon moving the cursor to the desired target, the user typically maintains the cursor at the 
acqu,red targe, while providing a "command gesture" associated with a physical acuon such as 
pressing a button, squeezing a trigger, depressing a pedal, or making some other gesture to command 
the execut.on of a particular opemting system function associated with the graphical object/target. In 
30 the preferred embodiment, the command gesture can be provided as other input 39 as shown m Figure 
1 For example, the "click" (press) of a physical button pos.tioned on a mouse or joy.stick while the 
cursor is on an icon allows an application program that is associated with the .con to execute. 
Likewise, the click of a button while the cursor is on a portion of a window allows the user to move 
or "drag" the window across the screen by moving the user object. The command gesture can be 
35 used to modify forces or for other functions in the present invention as well. For example, a button 
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on the user object can be designated to remove die forces applied in a certain region or target in GUI 



500. 



In other embodiments, the "command gesture" can be provided by manipulating the physical 
object of the interface device within provided degrees of freedom and/or with graphical objects 
displayed on the screen. For example, if a u.ser object has a third degree of freedom, such as linear 
translation along axis C of Figure 6. then movement in this direction can indicate a command gesture. 
In other embodiments, graphical objects on the screen can provide a command gesture when 
manipulated by a user. For example, if a pull-down menu is displayed, a small button can be 
displayed on or near each menu item. The user could then move the cursor onto the appropriate 
button to select that menu item. Also, a side view of button could be displayed, where the u.««:r moves 
the cursor into the button to "press" it and provide the command gesture. A spring force on user 
object 34 can be associated with this pressing motion to provide the feel of a mechanical button. 

The discussion below will build upon a concept of GUI targets being included in a particular 
hierarchy of levels in relation to each other. A first target that is included or grouped within a second 
target is considered a "child" of the second target and lower in the hierarchy than the second target. 
For example, the display screen 20 may display two windows. Windows are typically considered to 
be at the same hierarchy level, since windows typically arc not grouped inside other windows. 
However, a window that is grouped within a higher level window, such as a window included within 
a Program Manager window (see Figure 19). is considered to be at a lower level in the hierarchy than 
the grouping window. Within each window may be several icons. The icons are children at a lower 
hierarchy level than the window that groups them, since they are grouped within and associated with 
that window. These target concepts will become clearer below. It should be noted that one target 
may be displayed "within" or over another target and still be at the same hierarchy as the other target. 
For example, a window can be displayed within the outer perimeter of another window yet still not be 
grouped within that other window, .so that the windows have the same hierarchy level. 

The GUI permits the user to access variou.s operating system functions implemented by an 
operating system running u.. computer system 12. For example, the Windows operating system can 
be running on computer system 12 to implement operating system functions. Operating system 
functions typically include, but are not limited to, peripheral input/output functions (such as writing or 
reading data to disk or another peripheral), selecting and running application programs and other 
programs that arc independent of the operating system, selecting or managing programs and data in 
memory, viewing/display functions (such as scrolling a document in a window, displaying and/or 
moving a cursor or icon acro.ss the screen, displaying or moving a window, displaying menu titles 
and selections, etc.), and other functions implemented by computer system 12. For simplicity of 
discussion, the functions of application programs such as word processors, spreadsheets, and other 
applications will be subsumed into the term "operating system functions", ahhough the functions of 
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an application program are usually considered to be indeper^dent of the operating system. Typically, 
application programs make use of operating system functions to mterface with the user, for example. . 
word processor will implement a window function of an operating system to display a text file m a 
window on the display screen. An operating system function may typically be selected by the type 
of graphical object; for example, an icon may generally execute an application program, a window 
generally displays collections of other graphical objects, a slider bar scrolls images on the screen, a 
mem. item may perform a variety of operating system functions depending on its label, etc. 

In addition, other types of interfaces are similar to GUI's and can be used with the present 
invention For example, a user can set up a "page" on the World Wide Web which is implemented by 
a remote computer or ser^'er. The remote computer is connected to host computer 12 over a network 
such as the Internet and the Web page can be accessed by different users through the network, m 
page can include graphical objects similar to the graphical objects of a GUI. such as icons, pull-down 
menus, etc.. as well as other graphical objects, such as "links" that access a different page or portion 
of the World Wide Web or other network when selected. These graphical objects can have forces 
associated with them to assist in selecting objects or functions and informing the user of the graphical 
layout on the screen. In such an embodiment, the speed of data transfer between the host computer 
and a network node can often be slow. Therefore, the reflex embodiment as described above with 
reference to Figure 5 is quite suitable, since the local microprocessor 26 can implement force 
sensation processes controlled by commands received from the remote computer implememmg the 
Web page and/or from the host computer 12. In yet other embodiment.,, a simulated three- 
dimensional GUI can be implemented with the present invention, in which an isometnc or perspective 
view of a GUI environment and its graphical objects can be displayed. Alternatively, a "first person 
view of a GUI interface can be implemented to allow a user to select operating system functions 
within a simulated 3-D virtual environment. 

25 GUI 500 is preferably implemented on host computer 12 using program instructions. The use 

of program instructions to perform operations on a host computer and microprocessor is well known 
to those skilled in the art, and can be stored on a "computer readable medium," Herein, such a 
medium includes by way of example memory such as RAM and ROM coupled to host computer 12, 
memory 27. magnetic disks, magnetic tape, optically readable media such as CD ROMs. 

30 semiconductor memory such as PCMCIA cards, etc. In each ca.se. the medium may take the form of 
a portable item such as a small disk, diskette, cassette, etc.. or it may take the form of a relatively 
larger or immobile item such as a hard disk drive. 

In Figure 18. the display .screen 20 displays a GUI 500. which can. for example. l)e 
implemented by a Microsoft Windows® operating system, a Macintosh operating system, or any 
35 other available operating system incorporating a GUI. In the example shown, a program manager 
window 501 comains various icons 502 that are grouped by window 501, here labeled as "Main". 



20 
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"Startup", and "Tools", although other or different icons may be grouped within window 501. A 
menu bar 504 may be included in window 501 which permits pull-down menus to appear by selecting 
menu heading targets 505 with a user-controlled graphical object 506, such as a cursor, that is 
controlled by the user via a uscr-manipulabic device such as the user object 34. For example, a user 
may select any of the "File". "Options", "Window", and "Help" menu headings 505 to display an 
associated pull-down menu of menu items (shown in Figure 19). Typically, a command gesture such 
as a button press or other input 39 (as in Figure I) is also required to display a pull-down menu when 
cursor 506 is positioned at a menu heading 505. In alternate embodiments, a pull down menu might 
be automatically di.splayed (without a command gesture) when cursor 505 is positioned at the 
associated menu heading 505. In the subsequent description, the terms "user-controUed grapnical 
object" and "cursor" will be used interch^geably. 

The present invention provides force feedback to the user through u.scr object 34 based on a 
location, a velocity, an acceleration, a history of one or more of these values, and/or other 
characteristics of the cursor 506 within the GUI 500 enviromnent. Other "events" within the GUI 
mav also provide forces, as described above with reference to Figures 4 and 5. Several prcfeired 
embodiments of different forces or "force sensations" applied to the user object 34 are described 
below. AS described above in the embodiments of Figures 4 and 5. the host computer can provide a 
signal to local processor 26 (or directly to actuators 30) to apply different force sensations on user 
object 34. These "force sensations" can be forces of a single magnitude in one direction, or they may 
bean interaction or sequence of forces, for example, to create the sensation of a texture, a damping 
force, a barrier, etc. The terms "force" and "force sensation" (i.e. "type" of force) are used 
interchangeably herein, where it is assumed that single forces and/or sequences/interactions of forces 
can be provided. 

In one preferred embodiment of Figure 18. the force feedback depends upon a distance 
between cursor 506 and a target, such as window 501, using one of the aforementioned force models. 
The distance can be measured from one or more points within the window 501 or its perimeter. As 
depicted in Figure 18. the window 501 is considered to be the highest level target displayed in GUI 
500 (in actuality, the cmire screen area of GUI 500 is preferably considered the highest level target, as 
described below). Icons 502 and menu bar 504 arc targets that have a lower level in the hierarchy. In 
other situations, the window 501 could be grouped within with a higher level target, and the icons 
502 and menu bar 504 could include additional targets lower in hierarchy than the icons and menu 
bar. Alternatively, icons 502 and menu bar 504 can be the same hierarchical level as window 50 1 , if, 
for example, icons 502 were positioned outside of window 501 and were considered on the 
"desktop", i.e., not grouped in any particular window. In addition, none of the associated targets are 
resincted to be the same size or shape as their corre.sponding graphical objects, e.g. a target can be 
defined as a particular portion of a graphical object. 
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r,«l narddi^m is implemented by the GUI 500 and 
Herein it is assumed that a position control pardai^m ts imp.c 

2 r^tp^vidcd degrees of freedom of U,= o.robie«. Thu. when cursor ,06 ,s r^oved 
^ "o, user objec 34 is movtuE a corr.^«.i„g direcion. The dU,ance ma> user o^, c 
LT. no. ho s.nc distance >h« cursor 506 moves on screen 20. bu, is ryp.cal,y xl. d i^^ 
d funchon. When describing .he posi.ion of cursor 506 herein, ,he pos..,on of user 
:^«r^»i.hin provided degrees of freedom is assumed .„ correiare wi,h ,hc cursors pos^oa 
Wta orces are Lcribed herem as ■affecag-. 'innuencing" or -berng applied .o" cursor 506 . 

assumed dra. .hese forces are acu^iy being applred .0 user objecr 34 by acuarors 30. 
vi-hich in mm affects the position of cursor 506. 

,„ aitemate embodiments, a rate control par^gm can be used in GUI 500. For exarnplc^a 
user can push a joystick in one din=ction to cause .he cursor to move m tot dtreCton, wh re the 
Zhr7ioystikUovedin,hatdirection.thefaste,,he curst, wi,, move acrossthe screen (m^ 

:^menJ.ion of ra.e co„™.,. .n such an embodiment, for example, the user „^ mov * 
joysuck from *e origin p»i.ion and ..^ s.op moving «te joysiicU, and .he 
mLng across .he sceen a. a constant speed. Forces can be applied to user object 34 de end n. on 
the poLon of cursor 506 sitnilarly to the position conuol embodiment. Another example .here 
rcontro, paradigm v,„uld be appropriate ts ,ho button-like sue. or knob that ts pos.,»ne Mw« 
keys of the keyboard on many portable computers, and which uses rate control to move a cursor 
within a GUI. 

,„ a preferred embodiment, the host commands as described above with reference to Figures 
9- ,7 can be used to provide the various forces used for a GUI 500 environment. The -rellex mode 
of using the host computer 12 only for high-level su,.rvisory commands can be hclpfu, ,n mcrcasm 
th. response tim. for forces applied to the user object, which is essential .n creaung realts. cjd 
accurate force feedback. For example, i. may be convenient for host computer 12 to send a s^J 
representaUon" .o microprocessor 26, which ,s data describing the layout of all the l-'^^^ ^^ 
displayed in the GUI which are associated with forces and the types of these graphtcal ob,e ts 
Wd, page embodiment, the layoutAype of graphtcal objects can be downloaded from the .emo« 
computer providing the page). The microprocessor can store such a spatial represemauon m .r«n«>^ 
, .7 In addition, the microprocessor 26 can be prov.ded with ihe necessary .nstructtons or da« <o 
Correlate sensor readings with the positron of the cursor on the dtsplay screen. Tltc mic«>processor 
would then be able to check sensor readings, determine cursor and targe, positions, and determme 
ompu, forces independen.ly of hos, compu.er 12. The hos, could impletnem operaling sys.em 
funcions (such as d.splaytng images) when appropnare. and low-speed handshaking stgnals can be 
. commun,ca.edbe.ween processor 26 and hos. 12.0 correlaK *e microprocessor and hos. processes. 
Also memory 27 can be a pemianen. form of memoo' such as ROM or EPROM which srores 
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predetermined force sensations (force models, values, reflexes, etc.) for microprocc.-;.sor 26 that are to 
be associated with particular types of graphical objects. 

Other methods besides the use of the reflex embodiment can also be used to provide the forces 
within the GUI environmenl. For example, host computer 12 can be connected directly to sensors 28 
5 and actuators 30 of interface device 14 by a fast communication interface to control the force feedback 
on user object 34. thus eliminating the need for local microprocessor 26. 

In the described embodiment, targets such as window 501, icons 502 and menu headings 505 
have force fields as.sociated with them to influence and enhance the user's ability to move cursor 506 
to or around the targets. For example, icons 502 may have an attracUve force associated with them. 

10 This attractive force originates from a desired point I within each icon 502. which may be located at 
the center position of the icon. Alternatively, point I can be located at a different area of icon 502. 
such as near the perimeter of the icon. Likewise, window 501 preferably has an attractive force 
associated with it which originates from a point W within window 501 , which may be at the center of 
the window. Points 1 and W are considered to be "field origin points". Alternatively, force fields can 

15 originate from a point or region not shown on the screen. These atu^tive forces are known as 
"external forces" since they affect the cursor 506 when the cursor is positioned exiemiilly to the 
targets. Externa! and internal forces of targets are described in greater detail with respect to Figure 
20a. 

In alternate embodiments, the field origin need not be a point, but can be a region or other 
20 defined area. For example, as shown in Figure 18, the entire area of an icon 502a can be considered 
the "field origin region" for an attractive force. In such an embodiment, the cursor may be able to be 
moved freely in a ccitain dimension when within a region defined by the borders of the target. For 
example, if cursor 506 is in region Rl defined by the top and bottom borders of icon 502a. then 
horizontal forces might attract the cursor toward icon 502a. but no vertical forces would be applied. 
25 Similarly, if cursor 506 is in region R2 defined by the left and right borders of icon 502a. then only 
vertical attractive forces might affect the cursor. 

The attractive forces associated with window 501 and icons 502 are applied to user object 34 
to influence the movement of user object 34 and cursor 506, Thus, an attractive force associated with 
window 50 1 will cause host computer 1 2 to command the actuators 30 of interface device 1 4 to apply 

30 appropriate forces on user object 34 to move or bias the u.ser object. Forces are applied to user object 
34 in a direction such that cursor 506 is concspondingly moved in a direction toward field origin 
point W of window 501. It should be noted that the forces to user object 34 do not actually have to 
move the user object in the appropriate direction; for example, when using passive actuators, the user 
object cannot be physically moved by the actuators. In this case, resistive forces can be applied so 

35 that user object 34 is more easily moved by the user in the appropriate direction, and is blocked or 
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feels resistance when moving in other directions away from or tangent to point W (passive actuator 
embodiments are described in greater detail with respect to Figure 20c). The attractive force appUcd to 
user object 34. which would move or bias cursor 506 toward point W. is represented by dotted line 
507 in Figure 18. Preferably, the force is applied with reference to a single reference point of cursor 
506. which is the tip point T in the preferred embodiment. In alternate embodiments, the reference 
point can be located at the center or other location on cursor 506 or other user-controlled graphical 
object The attractive forces can be computed, for example, with a 1/R or 1/R2 relationship between 
field origin point W or I and cursor tip T to simulate gravity, as described above with reference to 
Figure 14. 

For other types of targets, repulsive fields may be associated with a field origin point. For 
example it mav be desired to prevem cursor 506 from moving to or accessing particular regions or 
targets on the screen within GUI 500. These regions might be displaying data thai is processing in 
the background or other data that is desired to not be selected by cursor 506. If window 501 is one 
such target, for example, a repulsive field in the opposite direction to that represented by line 507 can 
5 be associated with window 501 and can originate at field origin point W. The force would move user 
object 34 and cursor 506 away from the target, making it more difficult for the user to move cursor 
506 onto the target. 

In the preferred embodiment, the position of cursor 506 determines which field forces will, 
affect the cursor 506 and user object 34. As described in greater detail subsequently, targets 

0 preferably are associated with internal and external forces in relation to cursor 506. Preferably, 
attractive forces arc external forces and thus affect user object 34 and cursor 506 only when the cursor 
506 is positioned externally to the target. In the preferred embodiment, only the external forces of the 
highest level targets that arc external to cursor 506 will affect the cursor 506 and object 34. Thus, m 
Figure 18. only the attractive force of window 501 will affect cursor 506 and user object 34, since the 

>5 icons 502'and menu headings 505 are at a lower level in the hierarchy. If cursor 506 were positioned 
within window 501. only the attractive fields of icons 502 and menu headings 505 would affect 
cursor 506 and user object 34 and the attractive force 507 would preferably be removed. This 
relationship is described in greater detail with respect to Figure 20a. In alternate embodiments, the 
forces from various targets can be combined or excluded in different ways. 

30 FIGURE 19 diagrammadcally illustrates the GUI 500 wherein multiple windows 501. 530 

and 540 arc displayed on display screen 20. Grouped within window 501 are icons 502. menu bar 
504. window 518. and pull-down menu 517: window 518 includes an icon 519. Grouped within 
window 530 are icons 532 and menu bar 534. Window 540 includes icons 542 and menu bar 544. 

AU three windows 501. 530. and 540 are at the same hierarchical level. Therefore, in a 
35 preferred embodiment, when the cursor 506 positioned outside the perimeter of all three windows as 
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shown cursor 506 and user object 34 are influenced by a combination of the three external attracuve 
forces one attractive force from each window. These attractive forces are represented by dashed Imes 
(vectors) 520 522. and 524. Dashed line 520 represents the attractive force in a direction toward field 
origin point Wl of window 501 . line 522 represents the attractive force toward field origin pomt W2 
of window 530, and line 524 represents the attractive force toward field origin point W3 of wmdow 
540 The magnitudes of these forces are preferably dependent on a formula, such as the inverse of the 
distance between each target and point T of the cursor. These attractive forces are preferably summed 
together as vectors to provide a resulting total attractive force in a resultant direction having a resultant 
magnitude (not shown). Thus, cursor 506 and user object 34 would be moved or biased m the 
resulting direction until either reaching the resulting field origin point or until a condition occuned to 
change the forces applied to cursor 506. In alternate embodiments, other methods can be used to 
combine force vectors from multiple targets. For example, other organizations of hierarchies can be 
used Or the magnitudes of forces might not be summed, such that the resulunt attractive force can be 
assigned a predetermined magnitude or a magnitude that depends on the types of targets that have 
15 contributed forces. 

No forces associated with icons 502. 532. and 542. menu bars 504. 534. and 544. pull-down 
menu 510 imemal window 5 1 8, nor "thumb" and corresponding scroll bar 582 affect cursor 506 and 
user object 34 while cursor 506 is positioned external to the windows as shown. The principal task to 
be performed in Figure 19 is the activation or selection of a particular window, not a windows 
20 contents. Thus, the inclusion of forces arising from targets inside a window would interfere with 
window selection. Once the cursor 506 is positioned within a window, then the forces associated with 
the targets inside the window take effect. For example, once cursor 506 is moved within wmdow 501 
the external attractive force associated with window 501 is preferably removed and the external 
attractive forces of icons 502. window 5 18, and menu headings 505 are applied. The attractive force 
25 of icon 519 within window 518 is preferably not applied, since it is not at the highest hierarchy level 
external to cursor 506, i.e.. icon 519 is at a lower hierarchy level than window 518. 

Only forces associated with highest level external targets preferably affect cursor 506. One 
reason is that, if attractive forces associated with targets inside a window were added to the window's 
external force, then a window with several icons could "overpower" other windows by exerting a 

30 much greater magnitude of attractive force on user object 34 than the other windows. The cursor 506 
might then be trapped into always moving to the window with several icons. If each window affects 
cursor 506 equally, then it is easier for the user to move the cursor to the desired window. Of course, 
in alternate embodiments, if a window having more targets were desired to exert a greater force on 
cursor 506 than windows having less targets, then such an effect can be implemented. For example. 

35 the magnitude of the forces in such an embodiment could be limited so that the user would still be able 



RECTIFIED SHEET (RULE 91) 



wo 97/21 160 



-53- 



PCT/IB96/01441 



to select all of the windows displayed in GUI 500, yet the user would feel slightly stronger forces 
from windows having greater numbers of icons. 

The embodiment described above assumes that the magnitude of external force associated witli 
each window on cursor 506 is calculated the same way. However, in other embodiments, the 

5 magnitude of attractive or other forces associated with targets can differ depending on characteristics 
of the targets or can be commanded by the software programmer or user to be a desired magnitude. 
For example, the size of windows 501. 530. and 540 may determine the magnitude of atu-active force 
affecting cursor 506. If a user drags a window to be a smaller size, the attractive force associated 
with that window might be made proportionally smaller. For example, a virtual "mass" can be 

10 assigned to a target based on size, and the mass can be multiplied by the inverse of the distance 
between the target and the cursor to' equal the resulting attractive force. The cursor can also be 
assigned a mass, if desired, to .simulate real physical forces between objects. Also, other features or 
characteristics of the target, such as color, type, shape, etc., might control the magnitude of the force 
depending on how the programmer or user sets up a desired GUI force environment. 

15 In addition, a programmer of the GUI 500 or of an application program running under the 

GUI is preferably able to contfol the magnitude of the forces associated with particular targets 
displayed (or the "masses" of targets). For example, the force field host command and command 
parameters, described abos-e. may be able to designate a magnitude for particular displayed windows. 
Each target could thus have a different, predetermined force associated with it. This might allow a 

20 software developer to designate a desired force to be associated with a particular window for his 
application program running under GUI 500. In addition, in some embodiments, a user of GUI 500 
might be allowed to designate particular magnitudes of forces associated with targets. A menu 
command or other standard method to allow the user to associate forces with particular targets can be 
implemented. 

25 FIGURE 20a is a diagrammatic illustration of displayed targets illusU-aung the concepts of 

internal and external forces of the present invention associated with targets. As referred to herein, 
"external forces" are those forces associated with a target which affect cursor 506 when the cursor 
506 is positioned externally to that target, i.e. when the cursor positioned outside the perimeter of the 
target. In contrast, "internal forces" are those forces associated with a target which affect cursor 506 

30 when the cursor is positioned internally to the target, i.e., within the perimeter of the target. Each 
target preferably has external forces and internal forces assigned to it. as described below. Of course, 
the internal forces and/or external forces associated with a target may be designated as zero, 
effectively removing those forces. 

Target regions 550. 552. 554, 556. and 558 are displayed in GUI environment 500. Targets 
35 550, 552. and 554 are at the same hierarchical level, and are associated with graphical objects such as 
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windows or icons . Targets are "associated with" an appropriate graphical object such as ar. icon, 
meanine that they can be characterized as a property of the icon. The target is typically the same size 
as the alsociated graphical object, but may be defined to be smaller or large than the object, or to be a 
different shape than the object, in other embodiments. Targets 536 and 557 are grouped within target 
554 and are thus at the same hierarchical level as each other but at a lower hierarchical level than the 
other targets 550. 552. and 554. For example, targets 556 and 558 can be associated with icons, 
windows, menus, menu items within a menu 554. or other targets grouped within wmdow 5.4. 
Rectangular and circular targets are shown in Figure 20a. although other shapes, even inegulor ones, 
may be provided as targets. 

PoiMs 560. 562. and 564 represent possible locations of cursor 506 in GUI 500. As 
explained above with reference to Figure 18 and 19. external forces associ«ed «ith lower teve 
Jgets 556 and 558 wiU not affect cursor 506 when *e cursor is positioned =«e^ to mgher le^l 
target 554. Therefore, when cursor 506 is at point 560 cxtet^al to target regions 550. "^.^d » ■ 
Jtotal force on cursor 506 is e,ual to ^ sum of extetnal targe, forces assocate w th ach t^ 
550 552 and 554. As an exart-ple. the associated forces may be an attracuve or "P"'^'" 
ftem I described above. The forces would U^us be in a direction toward the T.e, ong.n po^n, W K 
W2. and W3 shown as dashed Unes 566, 567, and 569. Aitematively, the extern, forces can oe on 
or any combination of the force models previously described wi* respect to F.gures 9 and 14^ F 
exaile. a texture external force or a damping extern, force can. be applied, or a 
or other forces. Additionally, other forces and force models may be ass.gned as - 
should be noted that many types of force models do not re,uL-= a Held ong.n as tn the examples 
Figures 18 and 19. 

,„ addition, external target ranges are preferably ass.gned to each external --''''^ 
with each of me targets 550. 552. and 554. T*ese extern, ranges define - 
,3r,e. po^t P ,0 .he r^ge Mt in which the external force will be in effcc. In one -^"^--"^^ 
.x-get poin. o for defming ranges can be the same point as .he fe.d ong,n po,n.. a "own 0 - 
55 . For ex^ole, extem. range 555 may represent border of a defined ex,en,a. reg.on ^6 fo 
.arget 550. .Uc, is a prede„rm.ned distance from point P. If cursor 506 ^ P^'^'^ J"^ * 
external regton 568 from ^ perime,er of target 550 to external range 555, then ^ exter^. fore 
, Lltated With target 550 .s .n effect, „ cursor 506 is outside regton 568, then .e extern, force , 
notineffec. For example, the exten,. target force associated with target regton ^'' •^^^ITt 
560 because .ts extern, region 568 does not ext--nd to point 560, By def.n.ng su.h r=. =. a.e 
processing „me of local mrcroprocessor 26 and/or host computer 12 is ^^'^ 
orces need only computed and applied when ^ cursor is in these regtons, TI« = em. r^.ton 
5 568 can be defined as a distance from point P, or may .temattvely be defined w,.h respec. o tta 
perimeter of a targe, or may have a predetcmuned shape about its assoctated target re8,on, 
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addilion a total force resuU.ng from the external forces of muUipIe targets may have a ncwly- 
Iputed external range. In alternate cmbodimcnus, the regton outs.de the extern^ range ot a targe, 
can be assigned a different force model and/or magnitude .nstead of zero. 

Point 562 is located within target 554 (.ntemal to target 554) and externally to targets 556 and 
558 At pomt 562, the total force affectmg cursor 506 would be a combtnat.on of an mtemal target 
orce for arget 554 and external target forces for targets 556 and 558. Cursor 506 is "insulate from 
Telemal forces of targets 550, 552. and 554 since U is inside target 554. The external force 
a"d with targets 556 and 558 are similar to the external forces described above. The mternal 
L associated with target 554 affects cursor 506 only when the cursor is within the penmeter of the 
3 l^ lnternal target forces of the preferred embodiment are described below w.th reference to 

Figure 20b. 

Point 564 is intem. to target 556. A cursor 506 placed at point 564 would experience an 
internal force associated with target 556 and no other forces. There are no external forces affecun 
r 506 at this location, since there are no targets of lower hier^chical level grouped m target 
5 55rraddition.theinternalforceoftarget554.sremovedwhen 

force of a target of lower hierarchical level, which in this case is the internal force of target 556. 

FIGURE 20b is a diagrammatic illustration of a single target 570 which is associated w.th 
interna, and external forces. In the provided example, target 570 may be associated with a menu .tem^ 
button, icon, or window. An external region shape delineated by range 555 denotes a rcg.on o. ^ 
ZO external force associated with target 570. Cursor 506 is influenced by the extern, targe force f 
target 570 when it is inside the external region 568 def.ned between dashed Itne 572 and an outer 
perimeter 575 of target 570. Alternately, the external region can be defmed between dashc bne 57 
and an inner perimeter 577 of target 570. Recall that the target assorted wuh . graph.ca^ object need 
not be the same size and shape as the graphical object, so a target penmeter may he ms.de or outs.de 
25 the perimeter of the graphical object displayed on the screen 20. 

An internal target region 574 may include a dead region 576 and a capture region 578. 
region 576 .s defined as the innermost, central region of target 570 and extends to an inner penmeter 
577 In the dead region, forces associated with the dead reg.on ("dead region forces") apphed to 
cursor 506 would preferably be zero so as to allow substantially free movement of the cursor wtth.n 
30 th.s region (also, any external forces of any targets included within target 570 would be ,n effect). 
This dead region thus corresponds to the deadband regions discussed above with reference to F.gure.s 
9 and 14. as applied to the restoring and re.storing spring forces and the groove/d.vo. forces. 

Altemat.vely, a particular force or force model can be associated with dead region 576. For 
example, a damp.ng force or texture force sensat.on can be provided when the cursor is pos.ttoned 
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withm this region, provid.ng force feedback awareness to the user that cursor 506 is inside t^get 570^ 
Other force models can also be apphed. such a., the forces described above with respect to Figures 9 
and 14 In addition, the entire displayed GUI portion 500 on the screen 20 is preferably con.s.dered a 
target and a dead region force such as a damping force or a texture force can be applied to user object 
5 ^4 when pointer 506 >s moving over the background or desktop of the GUI. Such a dampmg force 
may greatly help users with a dexterity disability and allow these users to move pomter 506 more 
accuratelv Or individual windows can be assigned different dead region forces. This feature can be 
useful to distinguish the "feci" of different windows displayed on the screen, thus reducing the 
confusion of the user. For example, one window can have a texture dead region force of closely 
,0 spaced bumps, while a different window can have a texture dead region force °f >-P-^^ 
bumps. This allows the user to identify which window the cursor is in just by the teel of the dead 

region texture. 

The capture region 578 is preferably provided at or near the perimeter of target 570. The 
forces associated with capmre region 578 are applied to cursor 506 when the cursor is positioned 
15 within or is moved through the capture region. Since the capture region is typically narrow U may 
sometimes be difficult to determine if the cursor is within the capture region. For example, the host 
computer or local microprocessor 26 determines the location of cursor 506 (and user object 34 by 
taking samples of sensors 28. If the user is moving user object 34 very quickly, the -^mgs from 
the sensors may at too slowafrequencytoprovidedatashowing that the cunsorwaslocatedm^^^^^^^ 

20 capture region. The width of capture region 578 (i.e.. the distance from inner pcruneter 577 to out 
perimeter 575) can thus be n«de large enough so that the cursor can be detected within th= capture 
region even when the user moves the cursor quickly. Alternatively, a history of sensor readings can 
be checked to determine if the cursor was previously outside (or inside) the target 570. and if the 
cursor is subsequently inside (or outside) the target 570. thus indicating that the cursor has pas.sed 
25 through capture region 578 and that a capture force should therefore be applied to user object 34. 

In the preferred embodiment, two different forces can affect cursor 506. depending on 
whether the cursor has moved from the dead region to the external r.gion of the target (exiung target 
. 570) or vice-versa (entering target 570). When the cursor is moved from dead region 576 to external 
region 568, an "exit capture force" is applied to user object 34. The exit capture force is preferably a 

30 barrier or "snap over" force positioned at inner perimeter 577. which preferably includes a spring 
force as represented symbolically by springs 579 in Figure 20b. The spring force causes a spring 
resistance to the motion of cursor 506 in the exit direction, which starts as o .small resistive force m the 
direction toward the dead region 576 and which increases as the cursor is moved closer to outer 
perimeter 575. The spring force may cause the cursor/user object to move back toward dead region 

35 576 if the user lets go of the user object. This barrier force thus prevents the cursor from eas.^ 
"escaping" the target 570. In embodiments having passive actuators, a damping bamer force can be 
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provided instead of the spring force. The barrier force can be useful to keep cursor 506 wthm an 
icon scroll bar. or menu heading so that the user may more easily select the operation designated by 
the icon etc. In addition, by providing a zero dead region force and a barrier exit capture force, a 
user can move the cursor within the internal area of a target and "feel" the shape of the target. wh:ch 
adds to the sensory perception of graphical objects. Outer perimeter 575 of target 570 preferably 
defines a snap distance (or width) of the barrier, so that once cursor 506 is moved beyond penmeler 
575 the exit capture force is removed. The divot force mode! can be used when a capture force is 
desired on all four sides of the perimeter of target 570. and a groove force model can be used ,f 
capture forces arc only desired in one dimension. 

When the cursor 506 enters target 570. an "enti^' capture force" is applied to user object 34. 
Preferablv. the entry capture force is the same spring force as the exit capture force, in the saine 
direction 'toward the dead region 576. Thus, when cursor 506 f.r.t enters the capture region the 
spring force will immediately begin to push the u.ser object/cursor toward the dead region. The closer 
the cursor is positioned to the dead region, the less spring force is applied. In some embodiments, the 
magnitude of the enti>- spring force can be limited to a predetemuned value or offset to prevent the 
cursor 506 from moving past ("overshooting") target 570 due to excessive attractive force. 

Alternatively, an attractive force field similar to the external attractive force fields described 
above can be provided as the entry capture force. In such an embodiment, the direction of movement 
of cursor 506 must be established so that it is known whether to provide the exit capture force or the 
entr^ capture force. The history of sensory readings can be checked as described above to detemune 
cursor direction. In alternate embodiments, different or additional types of entry capture forces can be 
applied. 

In addition, a different "inema" force can be applied to user object 34 when cursor 506 is 
positioned in dead region 576 for particular types of targets and when specific conditions are met. 
For example, the inertia force can be applied when a command gesture, such as the prcssmg or 
holding of a button, is input by the user. In one preferred embodiment, the mertia force is provided 
when the user moves pointer -.06 into dead region 576. holds down a button on the joystick or 
mouse, and moves or -drags" the graphical object (and associated target 570) with pointer 506 across 
screen 20. The dragged target 570 has a simulated ' mass" that will affect the amount of inema force 
applied to user object 34. In some embodiments, the ineilia force can be affected by tiie velocity 
and/or acceleration of cursor 506 in addition to or instead of the simulated mass. Other factors that 
may affect the magnitude of ineitia force, such as gravity, can also be simulated. For example, if a 
large icon is dragged by cursor 506. then the user may feel a relatively large dumping force when 
moving user object 34. When the user drags a relatively small icon with pointer 506, then a smaller 
35 damping force should be applied to user object 34. Larger objects, such as windows, can be assigned 
different masses than other objects, such as icons. Alternatively, an icons mass can be related to 
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how large in terms of storage space (e.g. in byies) its associated program or fUe .s. For example, an 
icon of a large-sized f.le is more difficult to move (is "heavier") than an icon for a smaller-sized file. 
A target's mass can also be related to other target/graphical object characteristics, .such as the type of 
graphical object, the type of application program associated with the graphical object (i.e., larger mass 
5 for word processor icons, less mass for game program icons, etc.). or a predetermined pnomy level. 
Thus force feedback can directly relate information about a target to the user, assisting m performing 
and .selecting desired operating system tasks. In addition, an inertia force feature may be liseful .f a 
user wishes to retain a specific screen layout of graphical object., in GUI 500. For example, all the 
objects on the screen can be assigned a very high "mass" if the user does not want objects to be 
1 0 moved easily from the preferred layout. 

Other types of forces can also be applied to user object 34 when other command gestures are 
provided and/or when the target is dragged or moved, such as texture forces and jolts. In addition, if 
simulated masses are being used to calculate the external force of a target, as for the attractive gravity 
force described above, then that same mass can be used to compute an inertia force for the target when 
15 the target is dragged. In yet another embodiment, a target may have a spring foree associated wuh us 
position before it was moved. For example, when the user drags an icon, the movement of u.ser 
object 34 would feel like a spring is attached between the icon and its former position. This force 
would bias the cursor toward the former position of the icon. In a different, similar embodiment, a 
spring or other type of force can be provided on user object 34 when a graphical object is resized. 
20 For example, a window can typically be changed in size by selecting a border or comer of the 
v^.indow with cursor 506 and dragging the window to a desired size. If the window is dragged to a 
larger size, then a "stretching" spring force can be applied to the user object. If the wmdow is 
dragged to a smaller size, then a "compressing" spring force can be applied. Such spring lorces can 
also be provided ,n a CAD program when graphical objects are .stretched or otherwise manipulated. 
25 Tlie implementation of these types of forces can include a simple proportionality between 
displacement and force and is well known to those skilled in the art. 

Also, the targets for inertia forces can be defined separately from the targets for the internal 
and external forces as described above. For example, most windows in a GUI can only be dragged 
bv cursor 506 when the cursor is located on a "title bar" (upper portion) of the window or similar 
30 specific location. The window can he as,socialecl with a inertia target and a separate intemaiyextemiil 
force target. Thus, the target for the intemayextcmal forces can be defined to cover the entire 
window, while the target for the inertia forces can be defined to cover just the title bar of the window. 
If the cursor 506 were located on the title bar. then both inertia and internal forces could be in effect. 

In addition, damping and/or friction forces can be provided instead of or in addition to the 
35 inertia forces. For example, each graphical object can be assigned a simulated damping coefficient 
or a coefficient of friction. Such friction might be useful when free-hand drawing in a CAD 
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, . ■ ti,. ri IT where the coefficient of friction might be based on "pen size" of a 

app ication program in the GUI. where the coeiiicicni , . . . • ^ other 

drawing cuL. A texture force might also be apphed when a graphical object ts drugged. Other 
am e^^^^^^^^^ and associated graphical objects and functions include providing force joits or 
W when the cursor 306 encounters a region, when an object is released after havmg been 
dragged across the screen, when a window is entered or exited by the cursor, or when a winoow .s 
opened or closed. In a text document, these bumps can be provided when the cursor moves 
between words, lines, letters, paragraphs, page breaks, etc. 

FIGURE 20c IS a diagrammatic illustration of a target 559 in a GUI 500 providing a "groove- 
extern^ force. This type of external force is suitable for an interface device 14 ^-mg p^^^^ 
actuators 30 Passive actuators may only provide resistance to mouon of user object 34, and thu 
I provide an attractive or repulsive force field .s an external force of a target. Thus, an e.em. 
force of target5.59 can be provided as external grooves 561,e.g. the groove forcemodel as descn^^^ 

Love with reference toFigure Mean be used. These grooves are preferably positioned in honzo t. 
and vertical directions ^d intersect at the center C of target 559. It should be noted that grooves 56 
are preferably not displayed within GUI 5CX). :.d are shown in Figure 20c for explanato^ pu jse 
a e the grooves are felt, not seen). (Alternatively, the grooves can be displayed.) When curso 06 
is moved into a groove, resistive forces are applied to resist further movement ou. of the groove bu to 
Llv allow movement along the length of the groove. For example, if cursor 506 P°--;j 
horizontal groove 563a. the cursor 506 may freely be moved (..e. wUh no external ^^^^^ 
from target 559) left and nght as shown by arrows 565. However, the groove wal s prov.d 
resistive force to the cursor when the cursor is moved up or down. This tends to gu.de or bias the 
movement of the cursor 506 toward (or directly away from) target 559. Similarly, if cursor 506 is 
positioned at vertical groove 563b. the cursor may freely be moved up and down as shown by arrows 
557 but must overcome a resistive barrier force when moving left or right. The grcKivcs 561 
preferably have a predefined length which detemiines the external range of the external force of the 
target. 

When cursor 506 is moved along a groove toward the center of target 559. the cursor 
eventually reaches the center C of the target. At this position, both grooves 561 provide combined 
barrier forces to the cursor in all four directions, thus locking the cursor in place. Once the cursor is 
30 locked, the user can conveniently provide a command gesture to select the graphical object associated 
with target 559. In a preferred embodiment, the external groove forces are removed once the user 
selects the target. For example, if target 559 is associated with a button as shown in Figure 22. the 
cursor would be guided to target 559. and. once the button is selected, the grooves would be 
removed, allowing the cursor to be moved freely. Once the cursor moved ou. of the external region 
35 defined by the ends E of the groove.s, the external force would again be in effect. 
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FIGURE 21 is a diagrammatic illustration of display screen 20 showing GUI 500 and 
window 50 1 with a pull-down menu. The forgoing concepts and preferred embodiments will now be 
appned to selection of menu items in a GUI environment. Once the cursor 506 is inside the wm ow 
501 forces appUed to user object 34 depend upon the cursor 506 location relative to targe, wuhin 
window 501 on the next lowest level of the hierarchy below window 501. Menu bar .04 . 
preferably considered to be on the same hierarchy level as icons 502. so that both icons 502 and menu 
bar 504 exe. attractive external forces on cursor 506. Altemadvely. menu bar 504 - be asv.gned . 
hierarchy level below that of window 501 but above that of icons 502. wmch would allow onl> th.e 
" to attract cursor 506 (hierarchy levels of other ^aphic. objects might also be changed . 
Other embodimenis). 

Ficure 21 a=pl=>s window 50> wUh a fie puU-dow. menu 510. where «nu 5,0 includes one 
or .ore ;en. 1^516. The dispUy of »enu 5>0 resulrs fron, a ^'^'^^ '^J^^^ 
Kead^g 505 o. U,e menu bar 504, and is rypicaBy perfom«d by "-"^^ " . 

heading 505 and selecun, or holding down aburron, such as a mouse or pys„c. ^"-^ ° ^^J^^ 
down Lnu such as .he "Fiie" pull-down n,e„u 510 has been ^f'^^'^;^':::^^,,^, 
,h= menu 510 or .ts icems 5 16 will affec, cursor 506 and user objec, 34^ F , example . ^ 
506 is locared wi,mn window 501 as denored by .he dashed cursor oute 5 - . J =or 
..varurg d,e pull-down mer>u 510. a» cursor 506;user ob,ec, 3. . ^'^^^JZ^^ 
position a. ou.Une 5,2 .oward Held origin poin. S o,d,e menu .10 wUh - 
L .menu 5,0. .^«ma.ve,y, a field origin region defined as me en.ue m n. .10 b 
described above. Once .he cursor 506 rs ,.«.ed wrUun m= penme.e: of me u M a s , 
iccauon 514. *en *e a..ac.ve ex.e... force of *e menu ,s ,o longer ,„ e..cr An^ -m. _^ 
forces of menu 5,0, or menu ..ems 5,6, are .hen .n effec, as descnoed b'-'"-/ *^ 
has one ex,ema. force assoc,a,ed wi,h i. *a. a..ac.s cursor 506 .0 ^"^'"'^- '^^'^^ „„„ 
origin posiUon, of .he menu 510. Al.ema.iv.,y, each menu i.em " - ^ 
exremal force, wluch may al, sum ,0 a .o.a, force *a. can affec. ^ ^ J"^" 
ourside menu 5 10. For example, each menu ,.em magh. have ..s " ^ 

ownr.e,doriginpo..,oca.eda.*ecen,erofeachm=nu.^^^^^^^^ 
o*er embodunems. In add„ion, some menu ..ems 5 ,6 m:8h. b^ d-s^ n 

3 of grearer magm.de Oian o.her irerr.. Ex.ema, force ™gn.« ■ ^ » ' 
according .0 characerisucs of me menu ,. ems (s.2e, order m m. hsl, e.c.,, q 

iccordinz .o personal desires of a programmer or user of GUI 500. 

O.eposittonedinstdethepull-downmenu510.thecurso. 506 w^^^^^^^^^^^^^^ 
Of several menu items5l6demarcatedbyd.hedandsol^^^^^^^^^^^^^ 
^5 lines are typically not displayed in standard menus of GUIs but sh 

purposes. Preferably, the menu 510 has no internal forces, but each menu item 
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internal forces Wnich are in effect within the perimeters 52 1 of the item areas. The dashed imes cefine 
the perimeter of each menu item with respect to other menu items 516. The menu items art 
preferably similar to the target 570 shown m Figure 20b. Preferably, each menu item mcludes a zero 
ma..nitude force in its dead region 576 and mcludes a bamer or "snap-over" force (such as a spnng or 
dai^ping force) located at perimeter 521 as its exit capture force m accordance with tnat describee 
with reference to Figure 20b. This capture force keeps cursor 506 within a pamcular menu item 5 16 
once the cursor has moved there. In addition, each menu item 516 can mclude a "snap-to entry 
captui. force positioned at the middle of the menu item to attract the cursor 506 to this middle pomt. 
The snap-to force can be implemented as a groove force model along th. length of the menu item. 
Thus, the cursor is assisted in remaining withm a parr^cular menu item target, such a. the Open F ■ 
item taract 517. bv the use of force feedback as previously discussed with reference to Figure 20b. 
Each minu item 516 such as New. Open Move. Copy. etc. can have its own dead region or fr:e 
movement wu^n a item 5 16 and a capnare region to assist in keeping the cursor m the ^ 
target that it is located. Preferred force models are the grooves and barriers discussed with ref i.nce 
to Figure 14. For example, a groove force model can be provided at each menu item so that ext^ 
L e is necessary to move the cursor 506 "out" of the groove past a perimeter 521. but doc no 
prevent cursor 506 from moving left or right out of the menu. By impeding movemen b^cw.n 
selection areas 516. the force feedback prevents accident^ s^f.ng between menu "J 
th.e inadvertent selection of an incorrect menu item and operating system function, m menu em 
tvpiCly have no external force, since they abut at their borders. external force can be prov^ed a 
thlleftandnghtbordersofeachmenuitemif desired. The abovedescrrced "snap to 
be useful for snap-to grid lines in a CAD program or cor.s:rainins motion m a drawmg program 
perpendicular or 45-degree angle directions. 

I„ cto embodiments, other forces can be provided tn addition to those discussed to ease 
movement of cursor 506 over the menu ttems 5 16. For example, dte user may ""="">' ^^^J" 
cursor over some menu „ems 5 16 if a great deal of force has to be used .0 move the cursor 316 ove 
perimeters 521 between ...enu ttems. To prevent the undesired slapping over of 
L.p,ng force c^ be provided in the dead region 576 of each selecuon 5 16 to slow down the c rso 
. a L„u .tem. ^temadvely. a repulsive entry capture force c^ be provided by t« menu .tern * 
are not .mmedtately adjacent .0 the menu item that the cursor is in. such that the stapptng problem 
reduced. 

Tlte scroll bar or -sltder" 581 also preferably ts destgnated as a target of the present inventioa 
The scroll bar preferably includes a ".hamb" 380. a gu.de 382 .n whtch to move the thumb, and 
ZL 5S3. cLor 506 can be posittoned over thumb 580 .n the scroll bar 58, - 
and the user can scroll or move the view of icons. .e«. or other informatton shown m window ^1 b^, 
movtng thumb 580 in a vertical direction along guide 582. as ts well known to those stalled ,n the art. 
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Guide 58^ is preferably a target of the present mvention such that external forces and internal forces 
are associated with the guide. Preferably, an attractive external force is associated with the gu.de so 
that cursor 506 is attracted to a Held origtn pent N within thumb 580. Thus, the Held origin po.nt of 
the guide can vat^ its position within guide 582 when the user moves the thumb. The guide 58. can 
be designated the same hierarchical level as icons 502. or a higher or lower level. Internal forces of 
the guide are preferably equivalent to those of Figure 20b. The captun: forces on the top and bo.toi^ 
sides of the groove prevent cursor 506 from easily moving onto arrows 583 when movmg thumb 
580 In an alternate embodiment, the dead region of guide 582 has zero width, so that the cursor ts 
always attracted to a point halfway across the width of the guide. i.e. an entry capture force to the 
.Tuddle line L of the guide. This would be close to a groove force model, except that the sides of 
guide 582 near anows 583 would have a barrier force and thus be like a divot. In a pass:vc actuator 
(or other) embodiment, such a groove can be provided along guide 582. and the cursor can be locked 
onto thumb 580 as described with respect to Figure 20c. The cursor would, of course, st.ll be able to 
be moved with the thumb when locked on the thumb, and could be released with a command gesture. 

Preferably, thumb 580 and arrows 583 are considered children objects of guide 582. i.e.. the 
thumb and arrows are at a lower hierarchical level than the guide and are considered "w.th.n the 
guide. Thus, the external forces of the thumb and arrows are only applicable when cursor 306 is 
positioned within the guide. The external forces of arrows 583 are preferably zero, and thumb 580 
preferably has an attractive external force. H^e internal fores of thumb 580 and arrows 583 are 
preferably similar to those described with reference to Figure 20b. 

Thumb 580 can also be assigned incnia forces as described with reference to Figure 2 1 . m 
user could feel the mertia "mass" of the thumb when moving it along guide 582. Since thumb 580 
can be viewed as an .con w.th constrained movement, many forces attributable to .cons can be 
assigned to thumbs. 

As described above, graphical objects/targets such as icons 502 and window 501 can be 
assigned simulated "masses" which can be used to provide inertia forces when the targets are dragged 
across the screen. The .nertia forces can also be appUcd duo to collisions or other interactions with 
other graphical objects and targets. For example, if pointer 506 is dragg.ng icon 502. and the tcon 
collides with the edge 587 of window 501, then a coll.s.on force can be applied to u.ser object 34. 
This collision force can be based on the speed/direction of the icon/cursor as it was moved, the mass 
of the icon, and any simulated compliances of the icon 502 and the edge 587. For example, edge 587 
can be ass.gned to be very comoliani, so that when icon 502 is dragged into the edge, a spnng-l.ke 
force is applied to user objecl 34 which causes icon 502 and cursor 506 to bounce back away from 
edge 587. 
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AUematively. these same sort of "collision" forces can be applied to cursor 506 regardless of 
whether any object is being dragged or not. For example, certain edges, objects, or regions in GUI 
500 can either be designated as "pass-through" objects or as "solid" objects. Cursor 506 would be 
able to move over any pass-through objects without user object 34 feeling any forces. However, 
forces would be applied to user object 34 if cursor 506 moves over or into any solid object. Cursor 
506 could be assigned a mass of its own so that the user object will feel collision forces in accordance 
with the mass of cursor 506. the velocity of the cursor across the screen, and a assigned compliance 
of the cursor and the object moved into. This can be useful in a GUI to prevent or hinder access to 
ccnain objects or functions. Such objects could be designated as solid objects which would allow 
cursor 506 to be moved freely about the .screen without concern about selecting undesired functions. 



FIGURE 22 is a diagrammatic illustration of display screen 20 showing window 501 and a 
"pop-up" window 586. Window 501 includes icons 502. Window 586 includes buttons 584 and a 
"radio button" 585, and the window is typically removed from the screen after a button 584 has been 
selected. Buttons 584 can also be displayed in more "permanent" (i.e., non-pop-up) regions of GUI 
500. Similarly to the targets associated with the graphical objects described above, each button 584 in 
a window 586 in Figure 22 has external and internal forces associated with it, as described with 
reference to Figure 20a. Thus, an attractive external force (or other desired force) and a zero dead 
region force and divot capture force con be associated with each button 584. Essentially, the buttons 
SS4 are analogous lo menu items 516 in Figure 21. except that a certain distance on the .screen 
separates the buttons 584 from each other. Also, buttons 584 preferably have radially-shaped external 
region for their external forces. 

Radio button 586 is similar to buttons 586 in that a particular function may be .selected or 
toggled if the user moves cursor 506 onto the radio button 586 and provides a command gesture such 
as'pushing a button. Button 584 preferably is implemented similarly to buttons 584 except that button 
586 has a round perimeter and preferably a round external region. In other embodiments, buttons can 
. have other shapes. 

In an alternate embodiment, the forces associated with buttons 584 and 585 can be "turned 
off or otherwise changed after the buuon has been selected by the user using cursor 506. For 
example, an anractive external force and entry capture force of a button 584 can draw or guide the 
cursor to the button. The exit capture force impedes the cursor from moving outside of the button. 
Once the button is selected, however, the capture and external forces can be removed, so that the 
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cursor can be moved freely (and/or be affected by the forces associated with other targets on the 
screen). The forces can then be reapplied upon desired conditions. For example, once the cursor 
moves out of the external region of the button, then the forces would be back in cffea and would be 
reapplied when the cursor was moved back into the external region of the button. Likewise, some or 
5 all of the forces associated with the button could be changed to different types of force models once 
the button was pressed. This embodiment can aJso be applied to other types of graphical objects, 
such as icons, e.g., once the icon is selected, forces are removed until the cursor is moved out of the 
external region and back into the external region, when the forces would be reapplied. 

FIGURE 23 is a flow diagram illustrating a method 610 for providing force feedback within a 
10 graphical user mierfacc (GUI) environment beginning at a step 612. Initially, in a step 614. a position 
of the user object 34 is calibrated. This is accomplished so that an origin position for the user object 
can be dctcmiincd by host computer 12. In next step 614. forces are mapped or associated with 
graphical objects in the GUI. For instance, referring to the diagram of Figure 20a. external and 
internal target forces are associated with the targets 550. 552, 554, 556. and 558. More specifically 
15 referring to the example of Figure 19. the host computer associates types of graphical objects m GUI 
500 with external and internal forces. The mapping will generally include assigning one or more 
force models and range sizes/shapes to each external and internal region of types of graphical objects. 
For example, icons may be assigned panicular forces and ranges, and sliders may be assigned 
different forces and ranges. Also, particular icons or other objects can be assigned panicular forces or 
20 ranges if the programmer has so designated. If only a portion of a graphical object is to be used as a 
target, then that portion can be defined in this step. The process of mapping forces to graphical 
objects in the GUI is described in greater detail with respect to Figure 24. 

In step 618. the position of the user object 34 is read by host computer 12 and/or 
microprocessor 26 and the cursor position on the screen is updated accordingly. This is typically 

25 accomplished by first reading sensors 28 on interface device 14 to determine where user object 34 is 
positioned. These readings are then converted to the coordinates on screen 20 and the cursor is 
moved to the appropriate location corresponding to the position of the user object in a position conu-ol 
paradigm, as is well known to those skilled in the art. Since the sensor readings may include non- 
integer values, the sensor readings can be converted to integer values which are associated with 

30 coordinates on the screen so that the cursor position can be updated. However, when forces an: 
calculated (as in .step 622 below), the original non-integer .sen.sor readings are used, since these values 
include the necessar>' accuracy. 

In alternative embodimenLs, the display might be updated in other ways in response to the 
position or other characteristics of motion of the user object. For example, some application programs 
35 implemented by host computer 12 might use two dimensional, planar input to control other aspects of 
an interface or program, such as panning a screen, rotating a controlled object, moving a user- 
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controlled plavcr, vehicle, or viewpoint through simulated 3-D virtual space, etc. Also, the velocit>- or 
acceleration of the user object can be calculated and used as input. In other embodaments. Ln. 
mechanism 14 might allow three or more degrees of freedom to the user object, thus allowing other 
ways to control objects and operating system functions. 

In step 620. process 610 deteimines a target of lowest hierarchy m which the cursor is 
located As mentioned above in the discussion of Figures 1 8 -20a. the hierarchies assigned to targct.s 
influence the forces that are in effect on cursor 506. This process is described in greater detail w.th 
respect to Figure 25. In step 622. an appropriate force is determined from the external and intcmal 
forces for each target that affects the cursor, where the target selected in step 620 helps determine 
which forces are in effect. In addition, other conditions or events in the GUI may contribute to the 
forces applied to the user object. The contributing forces are combined and the combined total force is 
applied to the user object 34 by actuators 30. This step is described in greater detail with re.spect to 
Figure 26. After .step 622. the process returns to step 618 to again read the user object position and 
apply appropriate forces. 

FIGURE 24 is a now diagram illustrating an example of .step 616 of Figure 23. in which 
forces are mapped to graphical objects. The process begins at 630. and in step 632. an available 
target is selected to assign forces that target. After a target has been selected, process 616 implements 
a .series of steps 634. 636, 638. 640. and 642 to determine the particular target's type. These steps 
can be performed in any order, or simultaneously. Step 634 checks if the selected target is an icon. If 
so step 644 assigns a radial dead range, a radial capture range, and a radial external range to the icon. 
The "dead range" is the size of the dead region 576 about the center of the icon, defined by mncr 
penmeter 577 as shown in Figure 20b. The "capture range" is defined between inner and outer 
perimeters 577 and 575. so that a radial capture range indicates that the inner and outer perimeters ;irc 
circular about the center of the icon. The capture and external ranges are preferably radial even though 
the icon itself may be rectangular or shaped otherwise. In other embodiments, other shaped ranges 
can be assigned. The process then continues to step 652. described below. If the target is not an 
icon, the process continues to step 636. 

In step 636, the process checks if the selected target is a button or window; if so. step 646 
a.ss,gns rectangular dead and capture ranges and a radial external range to the selected target. Buttons 

30 are illu.strated in Figure 22. Since the windows and buttons arc rectangular, a rectangulai" capture 
range is desired to match the shape of the perimeter of the window or button. A radial external range 
can be provided as a predetermined distance from a center point of the window or button. The 
process then continues to step 652. If the target is not a button or window, the process continues to 
step 638. Step 638 checks whether the target is a radio button; if so. step 648 assigns radial internal 

35 and external ranges, since the radial button is typically circular in shape. The process then continues 
to step 652. If the target is not a radial button, the process continues to step 640. in which the process 
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checLs if the target is a slider. If so. step 650 assigns rectangular dead, capture, and external range^ 
to the guide, thumb, and arrows as explained previously. If the slider is implemented as a one 
dimensional groove, then the dead range would be linear, i.e.. zero in one dimension. The proces.. 
then continue, to ..tep 652. described below. If the target is not a slider, the process continues to step 

5 64^ where the process checks if the target is a menu item or menu heading (or a menu 5 10. in which 
preferably no internal ranges are assigned). If so. step 650 is implemented as described above, except 
that no external ranges are preferably assigned to menu items. In other embodiments, the process can 
test for other types of graphical objects to which the programmer wishes to assign ranges, if none of 
the steps 634 636 638. 640. or 642 are true, then control passes to step 643. in which the external 

10 and internal force ranges of the target arc set to zero. Alternatively, the process can check for a 
particular graphical object to which predetermined or desired force ranges are assigned. This special 
object can be designated as such by the programmer or user. If such a special object is provided, then 
the process can continue to step 652. 

After force ranges are assigned to the selected target in any of steps 644. 646. 648. or 650. 

IS step 652 determines whether the selected target is .special. If not, step 656 assigns force 
magnitudes and/or force models or force .sensation processes to the external and internal forces for 
the particular target according the target's type. For example, an icon may be assigned standard, 
predetermined force magnitudes or force models for its external attractive force and for its internal 
dead and capture forces. Alternatively, the object can be assigned a "mass" which will mfluence 

oo the magnitudes of the assigned forces. If the target is special, step 654 assigns any .special 
magnitude (or mass) to the target according to any particular instnictions or values provided by a 
programmer or user. This allows individual targets to be assigned desired force magnitudes. After 
cither step 654 or 656. method 6 1 6 ends at step 658. 

The assigned force ranges, magnitudes and models can also be .stored in memory 27 as a 
-JS "parameter page" by microprocessor 26 or host computer 12. For example, each parameter page 
might assign different types or ranges of forces to the graphical objects. These parameters pages 
can be loaded quickly to provide different force environments, or may allow host computer 12 to 
build another force environment by sending host commands while the processor 26 implements a 
different force environment. Parameter pages are described in greater detail with respect to U.S. 

30 patent application enliUed "Method and Apparatus for Controlling Force Feedback 

Interface Systems Utilizing a Host Computer." filed 12/1/95 on behalf of Rosenberg et al. 

FIGURE 25 is a flow diagram illustrating step 620 of Figure 23. in which the target of lowest 
hierarchy is determined in which the cursor resides. The process begins at 660. By well-known 
binary uee or set theoretic hierarchy methods, step 662 determines whether the cursor 506 is 
35 positioned w.thm the perimeter of a target and whether that target includes other children targets which 
the cursor is also within. For example, referring to Figure 19. process 620 may determine that the 
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cursor 506 is within window 501, but is also within window 518 of window 501. and that the cursor 
is additionally within an icon 519 of window 518. The targe, of lowest hierarchy of which the cursor 
was positioned would thus be the icon 519. 

Step 664 essentially determines whether the cursor 506 is in a region where two targets of the 
same hierarchical level overlap. This can occur if. for example, two icons or windows of the same 
(lowest) hierarchical level happen to be displayed on the same portion of the screen. Process 620 
quer.es whether the cursor 506 is in more than one of the lowest level targets. If the cursor 506 .s m 
an overlap region, then step 666 selects the "top" target whose object is displayed on .screen 20. Tlie 
"bottom- target will be partially or totally hidden by the top target. If there is no overlap in step 664. 
then step 668 selects the lowest level target nomially. The process is complete at 669 after .step 666 or 
668. 

FIGURE 26 is a flow diagram illustrating step 622 of Figure 23, in which an appropriate 
force is applied to the user object 34 based on the cursor's position and the target in which the cursor 
is located. The process begins at 670. Having detemiined the target of lowest hier^chical level in 

i which the cursor is positioned in step 620, step 672 calculates an internal force for that target 
containing the cursor 506 (the "lowest target"). The internal force is calculated using a force model or 
function, such as a force .sensation process, given appropriate parameters such as magnitude, 
duration, coefficients, sensor data, and timing data. Force models, force sensation proce.sses. and 
parameters were discussed above at length with respect to Figures 4-5 and 9-17. The internal force 

0 might be calculated in accordance with the dead region 576 if the cursor is positioned there; or, the 
internal force might be calculated according to a capture force if the cursor is positioned in capture 
region 574 or has j ust passed through the capture region. 

In step 674, a total force value is initialized to the internal force of the lowest target that was 
calculated in step 672. Thus, only the internal force of the lowest hierarchical target in which the 

•5 cursor is positioned is included in the total force that is to be applied to the user object. The mtemal 
forces of any higher level targets are preferably not included in the total force. As an example, 
consider a cursor 506 inside a window containing only icons. If the cursor 506 is not in an icon's 
target, the window itself is the lowest hierarchy target in which the cursor 506 resides. Only the 
internal target force for the window is calculated. If the cursor is moved into an icon, only the interna! 

30 force from that icon is included in the total force; the internal force of the window is ignored. 

Step 675 determines the children targets of the lowest target whose forces will affect the user 
object. These "external" children are included in the lowest target which the cursor is positioned in, 
but which are external to the cursor, i.e.. the cursor is not positioned in any of the external children. 
Thus, the external forces of the external children will affect cursor 506 and user object 34. Any 
35 targets included in the external children are preferably not added as a force. If the cursor is in the 
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"desktop" or background target of GUI 500. then the external children are the next highest level 
targets on the screen. For example, the windows 501 , 530 and 540 would be external children when 
cursor 506 is positioned on the desktop as shown in Figure 19. In alternate embodiments, the 
external children might also include additional lower level targets within other external children. 

,5 In .step 676. the process determines whether any external forces of the external children have 

not been combined into the total force. If so. step 677 selects a previously unvisited external child 
and computes the external force for the child. The external force from this child is only computed if 
cursor 506 is within the external range of the chUd; if the cursor is outside the external range, the 
external force is set at zero. This saves processing time if the cursor is not in the external range. 

10 Alternatively, if a particular force is assigned to regions outside the external range, that force is 
computed. The external force is computed according to the particular force model assigned to the 
external force, such as the attractive force field model described in the examples above. 

Step 678 computes the total force by adding the external force from the child of step 677 to the 
total force to be applied to the user object 34. It should be noted that the directions and magnitudes of 

15 the previous total force and the external force are taken into account when determining the dirccuon 
and magnitude of the resulting total force. For example, if the previous total force had a magnitude of 
5 in a left direction, and the external force had a magnitude of 8 in the right direction, then the sum of 
step 678 would result in a total force of magnitude 3 in the right direction. The process then returns to 
step 676 to check for another unvisited external child and add an external force to the total force in 

20 steps 677 and 678. Steps 676-678 are repeated until external force contributions from all the external 
children have been combined into the total force. 

After all the external children forces have been added \o loial force, then, from the negative 
result of .step 676, the process checks if a command gesture has been input by the user which would 
affect the force applied to the user object. For example, such a situation might occur if the inertia 

25 forces described above were implemented. These forces would be applied when the user held down a 
button or provided similar input and dragged an icon or window. If such input has been received, 
then the total force is adjusted based on the command gesture and the particular conditions or location 
of the cursor or other factor>i (such as the velocity of the cursor, mass of the dragged icon, simulated 
gravity, etc.) The "adjustment" to the total force may be an addition or subtraction to the magnitude of 

30 the iota] force and/or a change in direction, depending on how strong and in what direction the inertia 
force is applied. 

In next step 684, or after a negative result of step 680. the process checks of another condition 
affects the force on the user object is in effect. Such a condition, for example, might be when cursor 
506 collides with a "solid" graphical object of GUI 500 as discussed above, if such a feature is being 
35 implemented. The forces from such a collision would affect the total force output by actuators 30 on 



PCT/IB96/01441 

WO 97/21160 

-69- 

user object 34. If such a condition exists, then the total force is adjusted appropriately in step 686^ 
After step 686. or after a negative result of step 684, the total force ts applied to the user object 3. . 
step 688 using actuators 30 as explained previously. The process is then -mplete at 68^ 
alternative embodiments, steps 680-686 can be performed at other stages tn process 6... such 
5 before step 672. 

FIGURE 27 is a now diagram illustrating an example method 690 for applying internal or 
external forces to user object 34 from a stngle target, where cursor 506 is positioned near the target s 
boundar^ To stmpHfv the dtscussion. process 690 assumes that only one target is dtspla,. on 
screen 20 and thus does not take into account forces from other targets that may influence the force 
.0 pphed to the user object depending on the cursor's position. The steps of adding forces from 
multiple targets is described above with reference to Figure 26. Also, other necessary steps a.s 
Tetted above, such as updating the cursor position, are omitted from process 690 for exped.ency . 

Process 690 begins at step 692. and m step 694 determines whether cursor 506 is in a 
particular target's capture zone. If so. an optional step 696 determines whether the host compu^r 16 
15 and/or loc. r^croprocessor 26 last detected the cursor 506 in U>e target ead zone^ th. w. 
case then the cursor 506 is moving from the dead zone to the external zone. Thus, step 698 
applied. Where the exUing capture force is applied according to the appropriate force sens^ton 
process. For example, the exitmg capture force in the preferred embodimem ts a bamer such as 
sprmg force to prevent the cursor 506 from easily escaping the perimeter the target. The process 
20 then complete at 702. It should be noted that in the preferred embodimem. the exit and ent,^ capture 
forces are the same (a barrier force), so that step 694 is not necessao' in such an embod.ment. and 
steps 698 and 706 are the same step. Steps 694. 698. and 706 as shown are needed if the ent^- and 
exit capture forces arc different. 

If the last nonK:apture position of the cursor was not in the dead region, then the cursor is 
2«i most likely being moved from the external region of the target to the dead region of the target. this 
is the case, step 706 apphes the entry capture force to the user object 34 as described above wtth 
rcferen- • to Figure 20b. For example, in an alternate embodiment, the entry capture force can bean 
attractn-e force that pulls the cursor 506 and user object 34 toward the center of the target. The 
process is then complete at 702. 
30 If m step 694. the presem position of the cursor is not in the capture region, then the process 

checks if 'the cursor is in the dead region of the target in step 700. If so, then the internal dead region 
force assigned to the dead region ,s applied in step 701. In the preferred embod.ment. the dead region 
force is zero and thas step 701 is omitted; however, in other embodiments, a dead region force can be 
calculated based on a force sensation process such as a damping or texture force. The process .s then 
15 complete at 702. If the cursor is not in the dead region in step 700. then the process checks if 0^ 
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cursor is in the external region, as defined by the external range of the target, in step 703. If so, step 
704 applies the external force of the target to the user object. If the cursor is positioned outside the 
external range, then the process is complete at 702. Alternatively, if a force is assigned to the target s 
region outside the external range, then that force can be applied to the user object. 

5 The force feedback sensations of the present invention are advantageously provided in a GUI 

500 These forces can both assist a user in selecting and performing operating system function., and 
can inform the aser of the various graphical objects displayed by the GUI. In panicular, those users 
who suffer from spastic hand motion and other dextenty-debilitating conditions are greatly advantaged 
by the addition of these force feedback sensations in a GUI environment. Formerly difficult tasks 

10 such as maneuvering a cursor onto an icon become much easier usmg the force feedback of the 
presem invemion by implemenung attractive forces, damping forces, and other forces that assist a 
user in hand-eye coordination. 

While this invention has been described in terms of several preferred embodiments, it is 
contemplated that alterations, modifications and permutations thereof will become apparent to those 

15 skilled in the art upon a reading of the specification and study of the drawings. For example, many 
different types of forces can be applied to the u.ser object 34 in accordance with different graphical 
objects or regions appearing on the computer's display screen. Also, many vaneties of graphical 
objects in a GUI can be associated with particular forces to assist the user in selecting the objects or to 
notify the user that the cursor has moved into a particular region or object. In addition, many types of 

20 user objects can be provided to transmit the forces to the user, such as a joystick, a mouse, a 
trackball, a stylus, or other objects. Furthermore, certain terminology has been used for the purpo.ses 
of descriptive clarity, and not to limit the present invention. It is therefore intended that the following 
appended claims include all such alterations, modifications and permutations as fall within the mic 
spirit and scope of the present invention. 
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1. A method for providing force feedback to users interacting with a graphical user 
inierfact: environment of a computer system, the method comprising steps of: 

receiving an indication of movement of a physical object thai is manipulated by a user, said 
physical object being included in a human interface device that outputs said indication to said 
computer system; 

moving a user-controlled graphical object within a graphical user interface based on said 
indication of said movement of said physical object, wherein said user-controlled graphical object 
and said graphical user interface arc displayed on a display screen connected to said computer 
system, and wherein said graphical user interface allows said user to interface with operating 
system functions implemented by said computer system; 

outputiing an signal from said computer system to said interface device to command said 
interface device to apply a force sensation to said physical object, wherein said force is associated 
with at least one of said operating system functions of said graphical user interface. 

2. A method as recited in claim I wherein said user-conu-oUed graphical object is a cursor. 

3. A method as recited in claim 2 wherein said force sensation applied lo said physical 
object is determined, at least in part, by a location of said cursor in said graphical user interface 
with respect to targets located in said graphical user interface. 

4. A method as recited in claim 2 wherein said force sensation applied to said physical 
object is determined, at least in part, by a velocity of said cursor moving within said graphical user 
interface. 

5. A method as recited in claim 2 wherein said force sensation applied to said physical 
object is dcicnnined. at least in part, by an acceleration of said cursor moving within said graphical 
user interface. 

6. A method as recited in claim 2 wherein said force sensation applied to said physical 
object is determined, at least in part, by a history of previous locations of said cursor within said 
graphical user interface with respect to targets located in said graphical user interface. 
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7. A method as recited in claim 3 wherein said force sensation on said physical object 
assists said user to select said at least one operating system function associated with said force 
sensation. 

8 A inethod as recited in claim 3 wherein said force sensation on said physical object 
5 informs said user of other graphical objects in said GUI which can be manipulated to perform said 
at least one operating system function. 

9. A method as recited in claim 7 further comprising a step of performing one of said 
operating system functions as indicated by .said locauon of said cursor and as indicated by a 
command from said user, wherein said force sensation applied to said physical object assists said 

1 0 user in selecting said operating system function. 

10. A method as recited in claim 9 wherein said performed operating system function is 
associated with a particular type of target of said graphical user interface. 

1 1. A method a.s recited in claim 9 wherein a magnitude of said force sensation applied to 
said physical object depends on a particular target of said graphical user interface into which said 

15 cursor is moved. 

12. A method as recited in claim 9 wherein a type of said force sensation applied to said 
physical object depends on a particular target of said graphical user interface into which said cursor 
is moved. 

13. A method as recited in claim 9 wherein said target region includes an icon, and 
20 wherein said command from said user includes selecting a button on the interface device. 

14. A method as recited in claim 13 wherein said operating system function includes 
executing a program that is independent from said operating system, said program being associated 
with said icon selected by said cursor. 

15. A method as recited in claim 13 wherein said force sensation is an attractive force 
25 between said icon and said cursor, said attractive force being applied to said user object when said 

cursor is within a predetermined distance of said icon to assist said user in selecting said icon. 

16. A method as recited in claim 3 wherein said force sensalion is an obstructive force 
positioned at a boundary of a target to prevent said user from easily moving said cursor out of said 
target. 
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17 A method as recited in claim 7 wherein said force sensation is an inertia force applied 
to said physical object when a target region is moved by said cursor in said graphical user 
interface. 

18. A method as recited in claim 17 wherein sajd inenial force is a damping force thai 
5 damps movement of said user object according to a damping constant and the velocity of said 

cursor. 

19. A method as recited in claim 1 wherein said force sensation is a damping force applied 
to said physical object as said cursor is moved on said display screen. 

20. A method as recited in claim 1 wherein said physical object is a joystick handle. 

10 21. A method as recited in claim 1 wherein said physical object and said interface device 

are included in a mouse. 

22. A method as recited in claim 1 wherein said force sensation is applied to said physical 
object by an elecuically controlled actuator of said interface device. 

23. A method as recited in ciaim 2 wherein said signal output to said interface device is a 
15 high level host command sent to microprocessor local to said interface device and separate from 

said computer system,. 

24. A method as recited in claim 23 further comprising a step of providing layout data to 
said microprocessor, said layout data indicating a spatial layout of targets m said GUI such that 
sa:d microprocessor can check said spatial layout to determine when force .sensations associated 

20 with said targets may be applied to .said user object. 



25. A including program instructions for performing steps for a process of providing 
r.rce feedback to the user of a graphical user interface displayed by a computer system, the steps 
comprising: 

25 determining a location of a user-controlled cursor within a graphical user interface 

displayed on a display screen of a computer system, said cursor being controlled by said user by 
manipulating a physical object of an interface device; 

determining which targets displayed within said graphical user interface are a.ssociaied with 
target forces that affect said physical object based on said locauon of said uscr-controllcd cursor. 
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wherein said largets allow said user to interface with operating system functions implemenied by 
said graphical user interface; 

providing a signal to apply a resulting force to said physical object based on said forces 
associated with said targets which affect forces on said physical object. 

5 26. A computer readable medium as neciied in claim 25 wherein each of said targets are 

associated with at least two different target force sensations depending on said location of said 
cursor with respect to each of said targets. 

27. A computer readable medium as recited in claim 26 wherein a position control 
paradigm is implemented such that said location of said cursor in said graphical user interface 

10 approximately corresponds to a location of said physical object with reference to an origin of said 
interface device. 

28. A computer readable medium as recited in claim 27 wherein said two different target 
force sensations include an internal target force sensation and an external target force sensation, 
wherein said internal target force sensation is applied to said physical object when said cursor is 

15 located within said target, and said external target force sensation is applied to said physical object 
when said cursor is located outside said target. 

29. A computer readable medium as recited in claim 28 wherein said internal target force 
sensation includes a capture force sensation when said cursor is located near a boundary of said 
target, and wherein said internal target force sensation includes a dead region force sensation when 

20 said cursor is located near a center of said target. 

30. A computer readable medium as recited in claim 28 wherein said targets of said GUI 
arc ordered in a hierarchy, and wherein said step of determining which targets may affect force 
sensations on said physical object includes determining a cursor target of lowest hierarchy in which 
said cursor is Uxiated. 

25 3 1 . A computer readable medium as recited in claim 30 wherein an internal target force 

sensation associated with said cursor target affects, said physical object and wherein no other 
internal force sensations of other targets affect said physical object. 

32. A computer readable medium as recited in claim 31 wherein other targets at a 
hierarchical level lower than said cursor target have external force sensations that affect said 

30 physical object, wherein said cursor is not located in said other targets. 

33. A computer readable medium as recited in claim 32 wherein said resulting force 
sensation applied to said physical object is a sum of said internal target force sensation of said 
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cursor target and said external target force sensations of said other targets at said hierarchy lower 
than said lowest hierarchical level. 

34. A computer readable medium as recited in claim 33 wherein said other targets are 
included within said cursor target and are al a lower hierarchical level than said cursor target. 

35. A computer readable medium as recited in claim 32 wherein when said cursor is 
located in at least two of said cursor targets, only the target displayed on top of said other targets of 
said at least two targets is selected as said cursor target. 

36. A computer readable medium as recited in claim 25 further comprising a step of 
mapping force sensations to targets provided by said graphical user interface. 

37. A computer readable medium as recited in claim 36 wherein said step of mapping force 
sensations includes assigning ranges to said targets, said ranges defining external regions around 
said targets which affect said external force sensation on physical object when said cursor is 
located within said external regions. 

38. A computer readable medium as recited in claim 37 wherein said ranges are assigned a 

shape. 

39. A computer readable medium as recited in claim 38 wherein said step of mapping force 
sensations includes mapping internal and external target force sensations to said targets. 

40. A computer readable medium as recited in claim 29 wherein said capture force 
sensation includes a snap-over force sensation, said snap-over force sensation providing a 
resistance to said movement of said cursor when said cursor from within said target to outside said 
target. 

41. A computer readable medium as recited in claim 29 wherein said external target force 
sensation includes a force field, said force field providing an attractive force sensation to said 
physical object to draw said cursor toward said target. 

42. A computer readable medium as recited in claim 29 wherein said external target force 
sensation includes a force field, said force field providing a repulsive force sensation to said 
physical object to push said cursor away from said target. 

43. A computer readable medium as recited in claim 29 wherein said external target force 
sensation includes two intersecting groove force sensations, said groove force sensations biasing 
said cursor to move toward said target and locking said cursor on said target. 
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44. A computer readable medium as recited in claim 43 further comprising a step of 
removing said groove force sensations after said cursor is locked on said target when said target is 
selected by said user. 

45. A computer readable medium as recited in claim 36 wherein said target 1% an icon. 

46. A computer readable medium as recited in claim 36 wherein said target is a button. 

47. A computer readable medium as recited in claim 36 wherein said target is a slider. 

48. A computer readable medium as recited in claim 47 wherein said slider includes a 
thumb movable on a guide, and wherein an attractive force attracts said cursor to said thumb. 

49. A computer readable medium as recited in claim 36 wherein said target is a menu item 
on a pull-down menu. 

50. A computer readable medium as recited in claim 36 wherein said target is a window. 

5 1 . A computer system for providing force feedback to a user of a graphical user interface 
displayed by said computer system, comprising: 

means for determining a location of a u.ser-controlled cursor within a graphical user 
interface displayed on a display screen of a computer system, said cursor being controlled by said 
user by manipulating a physical object of an interface device, and wherein target., are provided in 
said graphical user interface which are associated with operating system functions implemented by 
said graphical user interface, said cursor being movable to select said targets to select said 
associated operating system function; 

means for determining which targets displayed within said graphical user interface are 
associated with target forces affecting said physical object based on said location of said user- 
controlled cursor; 

means for providing a signal to apply a resulting force sensation to said physical object 
based on said forces associated with said targets which affect forces on said physical object. 

52. A computer system as recited in claim 51 wherein said means for determining said 
targets includes means for determining a cursor target of lowest hierarchy in which said cursor is 
located. 

53. A computer system as recited in claim 52 wherein said cursor target is a menu item in a 
pull-down menu. 
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J wherein said menu item is associated with a 

54 A computer svstem as recited in claim 5 J wncrein saja mwiu 

^ 'a f ^ item said snap-over force providing a obstructive 
snap-over force at a boundary of said menu item, saiu snup-u f 

r u ;h riirsor is moved from one menu item to another 

force to said movement of said cursor when said cursor is movca iron 

menu item. 

5. A computer svstem as recited in claim 53 wherein said menu item is associated with a 
snap-to force at a center a'rea of said menu item, said snap-to force providing on attractive force that 
keeps said cursor at said center area of said menu item. 

56. A computer system as recited in claim 52 wherein said cursor target is a slider having a 
thumb that can be moved within a linear guide. 

57 A computer system as recited in claim 56 wherein said slider is associated with a snap- 
to force provided at a middle line along a length and halfway across a width of said guide o said 
slider. s-L snap-to force providing an attractive force that keeps said cursor at said middle hne of 
said guide. 

58. A computer system as recited in claim 52 wherein said cursor target is an icon. 

59 A computer system as recited in claim 56 wherein said icon is associated with an 
attractive force associated with a surrounding region having a predetemuned size, said attiBCUve 
force pulling said physical object and said cursor when said cursor is moved within said region. 

60 A computer system as recited in claim 59 wherein said icon is associated with an 
capture force associated with a boundary of said icon, said capture force providing an obstructive 
force to said physical object when said cursor is moved out of .said icon. 

6 1 A computer system as recited in claim 59 wherein said icon is associated with a dead 
region force associated with an internal region of said icon, said dead region force providing a 
predetermined damping force to said physical object when said cursor is moved in said internal 



region. 



15 62. A computer system as recited in claim 52 wherein said target is a graphical button. 

63. A system for providing force feedback to a user manipulating an interface device, the 
system comprising: 

a host computer svstem for receiving an input control signal describing a location of a user 
manipulable object in a degree of freedom and for providing a host output control signal, wherein 
30 said host computer system updates the location of a user-controlled cursor within a graphical user 
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interface displayed on a display screen of said host computer system, said cursor being updated 
based on said input control signal; 

a processor local to said interface device and separate from said host computer system for 
receiving said host output control signal from said host computer system and providing a processor 
output control signal; 

an actuator for receiving said processor output conu-ol signal and providing a force along 
said degree of freedom in accordance v^ith said processor output control signal to said user 
manipulable object coupled to said actuator; and 

a sensor for detecting motion of said user manipulable object along said degree of freedom 
and outputting said input control signal including information representative of the position and 
motion of said object. 

64. A system as recited in claim 63 wherein said sensor outputs said input control 
signal to said processor, and wherein said processor provides said input control signal to said host 
computer. 

65. A system as recited in claim 63 u'herein said host computer system displays a 
number of targets in said graphical user interface and determines which targets are associated with 
target forces that affect said physical object based on said location of said user-conu-olled cursor, 
wherein said targets allow said user to interface with operating system functions implemented by 
said graphical user interface. 

66. A system as recited in claim 65 wherein said host output conu-ol signal commands 
resulting force to said physical object based on said forces associated with said targets which affect 
forces on said physical object. 

67. A system as recited in claim 66 wherein said processor is operative in a force 
sensation process to provide said processor output control signal to said actuator ii. response to 
said position and motion of said object independently of said host output control signal. 

68. A system as recited in claim 67 wherein said host output signal is a high level 
command from said host computer system, and wherein said processor implements one of a 
plurality of local processes selected in accordance with said high level command to implement said 
force sensation process. 

69. A system as recited in claim 65 wherein said user manipulable object can be moved 
by said user in a plurality of degrees of freedom, and wherein said system further comprises an 
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actuator for providing a force along a degree of freedom of said object, and a sensor for detecting 
motion of said object in said degree of freedom for each of said plurality of degrees of freedom. 

70 A system as recited in claim 65 further comprising a serial interface for outpuning 
said host output control signal from said host computer system and for receiving said input control 
signal to said host computer system. 

71. A system as recited in claim 70 wherein said serial interface includes a Universal 
Serial Bus. 

72 A system as recited in claim 65 further comprising a clock coupled to said 
processor, wherein said processor accesses said clock to determine, in part, said force provided by 
said actuator. 

73 A system as recited in claim 65 further comprising a memory device accessible by 
said local processor, wherein said local processor stores a spatial representation of said targets m 
said memory device such that said processor may detect when said target forces affect said physical 
object. 

74. A system as recited in claim 73 wherein said memory device includes a permanent 
memory storage for storing predetermined target forces associated with said targets. 

75. A method for providing force feedback for graphical objects in a game implemented on 
a computer system, the method comprising the steps of: 

displaying a user-controlled first graphical object on a display screen of a computer system, 
said graphical' ob,ect moving on said display screen during a game in response to manipulations of 
a physical object of an interface device by a user, said interface device being coupled to said 
computer system; 

displaying a second graphical object on said display screen; 

when said first graphical object collides with said second graphical object on said screen: 

i a) displaying a compression of said first object where said second objeci contacts 

said first object, wherein said first object has a predetermined simulated compliance, and said 
second object has a predetermined simulated mass; 

b) outputting a force command to said interface device to apply a force to said 
physical object manipulated by said user in at least one degree of freedom provided by said 
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interface device, said force being applied in the direction of said compression and having a 
magnitude in accordance with said simulated masses of said first and second graphical objects. 

76. A method as recited in claim 75 wherein said force has a magnitude also in accordance 
with a simulated compliance of said first graphical object. 

77. A method as recited in claim 76 wherein said force has a magnitude also in accordance 
with simulated velocities of said first graphical object and said second graphical object. 

78. A method as iccitcd in claim 76 wherein said force has a magnitude also in accordance 
with a simulated gravity acting on said objects. 

79. A method as recited in claim 76 wherein said first graphical object is a linear segment, 
and wherein said second graphical object is a circular object. 

80 A method as recited in claim 76 wherein said second graphical object moves on said 
display screen during a game in response to manipulations of a second physical object of an second 
interface device by a second user, said second interface device being coupled to .said computer 
system. 

81 A method as recited in claim 80 wherein said first and second interface devices input 
sensor signals indicating motion of said first physical object and .said second physical object, 
respectively, wherein said user is a first user and said force command to said first interface device 
is based on sensor signals output from said second interface device, and wherein a force command 
output to said second interface device is ba.sed on sensor signals output from said first interface 
device. 

82. A method as recited in claim 76 wherein said second graphical object moves on said 
display screen dunng a game in response to manipulations of a second physical object of an second 
interface device by a second user, said second interface device being coupled to a second computer 
system coupled to s;: i computer system through a network interface. 

83. A method as recited in claim 76 further comprising a step of changing said simulated 
compliance of said first graphical object when an input command is received from said user 
operating an input device on said interface device. 

84. A method as recited in claim 76 further comprising displaying a goal object on said 
display screen and wherein said user moves said first graphical object to block said second 
graphical object from moving into said goal object. 
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85. A method as recited in claim 79 further comprising changing a displayed orientation of 
said linear segment according to input data received from said user operating said input device. 

86 A method as recited in claim 85 wherein .said physical object is a joystick handle, and 
wherein said input data received from said user includes a spin of said joystick handle in a rotary 
degree of freedom. 

87. A method a.s recited in claim 86 wherein said interface device provides tuo linear 
degrees of freedom to said joystick handle in addition to said rotary degree of freedom. 

88. A method for providing force feedback for interacting simulated objects in a 
simulation implemented on a computer system, the method comprising the steps of: 

displaying a user-controlled fir.<;t simulated object and a second simulated object on a 
di.splay device of a computer system, said first simulated object moving on said display device 
during a simulation in response to manipulations of a physical object of an interface device by a 
user, said interface device being coupled to said computer system; 

determining when said first simulated object engages said second simulated object within 

said simulation; 

displaying said determined engagement of said first simulated object with said second 
simulated object, wherein said first simulated object has a predetermined simulated compliance 
and said second object has a predetermined simulated mass; and 

outputting a force command to said interface device to apply a force to said physical object 
manipulated by said user in at least one degree of freedom provided by said interface device, said 
force being applied in the direction of said engagement of said second simulated object with said 
first simulated object and having a magnitude in accordance with said simulated mass of said 
second simulated object. 

89. A method as recited in claim 88 wherein a simulated compliance of said first 
simulated object additionally affects said magnitude of said force. 

90. A method as recited in claim 89 wherein simulated velocities of said first simulated 
object and said second simulated object additionally affect said magnitude of said force. 

91. A method as recited in claim 89 wherein a simulated gravity acting on said simulated 
objects additionally affects said magnitude of said force. 
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92. A method as recited in claim 88 wherein said first simulated object elongates at a point 
of impact with said second simulated object in accordance with said simulated compliance of said 
first simulated object. 
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